Sep 17 23:06:14.244771 ip-10-0-137-236 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 23:06:17.705290 ip-10-0-137-236 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 23:06:17.705290 ip-10-0-137-236 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 23:06:17.705290 ip-10-0-137-236 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 23:06:17.705290 ip-10-0-137-236 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 23:06:17.705290 ip-10-0-137-236 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 23:06:17.706081 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.705349    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 23:06:17.710937 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710916    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:06:17.710937 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710932    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:06:17.710937 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710937    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:06:17.710937 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710940    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:06:17.710937 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710943    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:06:17.710937 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710945    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710949    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710952    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710956    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710958    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710963    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710968    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710971    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710973    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710976    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710979    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710982    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710985    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710987    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710990    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710993    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710995    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.710998    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711001    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:06:17.711155 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711003    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711006    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711011    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711014    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711017    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711019    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711022    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711024    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711026    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711029    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711031    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711034    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711038    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711042    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711045    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711049    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711052    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711055    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711057    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711060    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:06:17.711627 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711062    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711065    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711068    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711070    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711073    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711075    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711078    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711081    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711083    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711086    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711088    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711091    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711094    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711097    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711100    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711102    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711105    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711108    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711110    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:06:17.712126 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711113    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711115    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711118    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711120    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711123    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711125    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711128    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711130    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711135    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711138    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711140    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711143    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711145    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711148    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711151    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711155    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711158    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711161    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711164    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711168    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:06:17.712686 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711172    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711174    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711177    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711179    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711182    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711185    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711187    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711190    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711192    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711195    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711198    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711200    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711202    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711205    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711207    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711210    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711895    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711902    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711905    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711908    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:06:17.713176 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711911    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711915    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711917    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711920    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711923    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711926    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711929    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711932    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711935    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711937    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711940    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711943    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711946    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711949    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711952    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711955    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711959    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711963    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711967    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711970    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:06:17.713673 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711973    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711975    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711978    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711980    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711983    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711986    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711989    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711991    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711994    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.711997    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712000    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712003    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712007    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712011    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712015    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712018    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712021    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712024    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712026    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:06:17.714181 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712029    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712032    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712035    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712039    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712041    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712046    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712049    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712052    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712055    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712057    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712060    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712062    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712065    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712068    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712071    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712073    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712076    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712079    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712081    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712084    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:06:17.714677 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712087    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712089    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712092    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712094    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712097    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712100    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712103    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712106    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712109    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712111    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712114    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712117    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712119    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712122    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712124    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712127    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712130    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712132    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712135    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712138    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:06:17.715164 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712141    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712144    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712146    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712148    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712151    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712153    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712156    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712159    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712161    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712164    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712166    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712169    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712171    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712174    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712176    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712179    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712545    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712551    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712554    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712557    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:06:17.715674 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712560    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712563    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712566    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712568    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712571    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712574    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712576    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712579    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712581    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712584    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712587    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712589    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712593    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712596    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712598    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712601    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712603    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712606    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712608    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:06:17.716156 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712611    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712614    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712617    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712619    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712622    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712625    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712627    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712630    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712633    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712635    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712638    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712641    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712643    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712645    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712648    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712651    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712653    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712656    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712658    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712660    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:06:17.716624 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712664    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712666    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712669    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712671    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712674    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712676    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712680    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712685    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712688    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712691    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712694    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712696    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712699    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712702    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712705    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712708    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712710    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712713    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712716    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712720    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:06:17.717183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712724    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712727    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712730    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712732    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712735    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712738    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712741    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712744    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712749    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712752    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712755    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712758    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712760    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712763    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712765    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712768    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712770    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712773    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712775    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:06:17.717693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712781    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712784    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712786    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712789    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712791    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712794    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712797    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712799    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712802    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712804    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712807    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712810    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712812    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712815    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712817    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712820    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.712823    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712890    2558 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712923    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712930    2558 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712934    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 23:06:17.718165 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712938    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712942    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712946    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712950    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712953    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712956    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712959    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712963    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712966    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712969    2558 flags.go:64] FLAG: --cgroup-root=""
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712972    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712976    2558 flags.go:64] FLAG: --client-ca-file=""
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712980    2558 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712983    2558 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712988    2558 flags.go:64] FLAG: --cluster-domain=""
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712990    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712993    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.712996    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713000    2558 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713003    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713006    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713009    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713012    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713016    2558 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 23:06:17.718695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713018    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713021    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713025    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713028    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713032    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713035    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713037    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713040    2558 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713043    2558 flags.go:64] FLAG: --enable-server="true"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713046    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713051    2558 flags.go:64] FLAG: --event-burst="100"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713054    2558 flags.go:64] FLAG: --event-qps="50"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713057    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713060    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713063    2558 flags.go:64] FLAG: --eviction-hard=""
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713066    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713069    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713072    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713077    2558 flags.go:64] FLAG: --eviction-soft=""
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713081    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713083    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713087    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713090    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713093    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713096    2558 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 23:06:17.719266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713099    2558 flags.go:64] FLAG: --feature-gates=""
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713102    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713105    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713108    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713111    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713114    2558 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713117    2558 flags.go:64] FLAG: --help="false"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713120    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-137-236.ec2.internal"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713123    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713126    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713129    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713132    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713136    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713139    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713142    2558 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713144    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713147    2558 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713150    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713153    2558 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713156    2558 flags.go:64] FLAG: --kube-reserved=""
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713159    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713163    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713166    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713168    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 23:06:17.719907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713171    2558 flags.go:64] FLAG: --lock-file=""
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713175    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713179    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713183    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713188    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713193    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713196    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713199    2558 flags.go:64] FLAG: --logging-format="text"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713202    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713205    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713208    2558 flags.go:64] FLAG: --manifest-url=""
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713210    2558 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713215    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713218    2558 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713221    2558 flags.go:64] FLAG: --max-pods="110"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713224    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713227    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713230    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713233    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713236    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713239    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713242    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713249    2558 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713252    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713255    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 23:06:17.720496 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713258    2558 flags.go:64] FLAG: --pod-cidr=""
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713261    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713266    2558 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713269    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713272    2558 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713275    2558 flags.go:64] FLAG: --port="10250"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713278    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713280    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04f558ea41563bfb0"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713286    2558 flags.go:64] FLAG: --qos-reserved=""
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713289    2558 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713292    2558 flags.go:64] FLAG: --register-node="true"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713295    2558 flags.go:64] FLAG: --register-with-taints=""
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713299    2558 flags.go:64] FLAG: --registry-burst="10"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713302    2558 flags.go:64] FLAG: --registry-qps="5"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713305    2558 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713308    2558 flags.go:64] FLAG: --reserved-memory=""
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713311    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713314    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713317    2558 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713320    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713323    2558 flags.go:64] FLAG: --runonce="false"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713326    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713328    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713331    2558 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713334    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 23:06:17.721149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713337    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713340    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713343    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713345    2558 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713348    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713351    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713354    2558 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713356    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713359    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713362    2558 flags.go:64] FLAG: --system-cgroups=""
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713365    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713370    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713373    2558 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713376    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713379    2558 flags.go:64] FLAG: --tls-min-version=""
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713383    2558 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713386    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713389    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713392    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713411    2558 flags.go:64] FLAG: --v="2"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713416    2558 flags.go:64] FLAG: --version="false"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713421    2558 flags.go:64] FLAG: --vmodule=""
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713425    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713428    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713561    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:06:17.721779 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713565    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713568    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713571    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713574    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713577    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713580    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713582    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713585    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713588    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713590    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713593    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713595    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713598    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713600    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713603    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713605    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713608    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713611    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713614    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713616    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:06:17.722392 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713619    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713621    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713625    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713628    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713631    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713633    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713636    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713640    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713644    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713648    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713652    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713655    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713659    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713662    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713665    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713668    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713670    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713673    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713675    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:06:17.722896 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713678    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713681    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713684    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713687    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713690    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713692    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713695    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713697    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713700    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713702    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713712    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713715    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713717    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713720    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713723    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713729    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713732    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713734    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713737    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:06:17.723361 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713740    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713742    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713745    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713748    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713751    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713753    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713756    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713759    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713761    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713764    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713766    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713769    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713771    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713774    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713777    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713779    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713782    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713784    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713787    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713790    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:06:17.723845 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713792    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713795    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713797    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713800    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713802    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713805    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713807    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713810    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713813    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713818    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713820    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713823    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713826    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713829    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713832    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713834    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713837    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713841    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713844    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:06:17.724356 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.713846    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.713851    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.720947    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.720961    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.720981    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.720989    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721043    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721047    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721051    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721054    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721058    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721060    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721063    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721066    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721069    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:06:17.724820 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721072    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721074    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721077    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721080    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721082    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721086    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721089    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721093    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721096    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721098    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721101    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721104    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721107    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721109    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721112    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721115    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721118    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721121    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721123    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:06:17.725183 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721126    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721130    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721132    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721135    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721138    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721141    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721144    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721146    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721149    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721152    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721155    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721158    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721162    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721165    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721169    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721172    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721174    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721177    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721180    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:06:17.725731 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721182    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721185    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721187    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721190    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721193    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721195    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721198    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721200    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721203    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721205    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721209    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721211    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721214    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721217    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721220    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721223    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721226    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721228    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721231    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721234    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:06:17.726202 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721237    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721239    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721242    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721244    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721247    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721249    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721252    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721254    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721257    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721260    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721262    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721265    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721267    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721270    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721272    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721275    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721278    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721280    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721284    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:06:17.726709 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721287    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721291    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721294    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721296    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721300    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721303    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721306    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721309    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721312    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721314    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721317    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721319    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721322    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.721327    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721428    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:06:17.727163 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721432    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721435    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721438    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721441    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721444    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721446    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721449    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721453    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721457    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721460    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721462    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721465    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721468    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721471    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721474    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721477    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721480    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721482    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721485    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721487    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:06:17.727552 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721489    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721492    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721495    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721498    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721501    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721503    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721506    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721508    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721528    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721531    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721534    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721536    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721539    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721542    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721544    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721547    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721549    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721552    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721555    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:06:17.728032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721558    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721561    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721564    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721566    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721569    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721572    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721575    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721577    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721580    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721583    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721585    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721588    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721590    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721593    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721595    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721598    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721601    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721604    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721606    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:06:17.728508 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721609    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721611    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721614    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721616    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721619    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721621    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721624    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721626    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721629    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721633    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721636    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721639    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721642    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721645    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721648    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721650    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721653    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721655    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721658    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721660    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:06:17.728980 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721663    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721666    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721668    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721671    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721673    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721676    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721678    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721681    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721683    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721686    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721689    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721692    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721695    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721697    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721700    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721702    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721704    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721707    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721709    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:06:17.729455 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:17.721712    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:06:17.730048 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.721717    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 23:06:17.730048 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.721809    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 23:06:17.730048 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.723871    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 23:06:17.730048 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.724818    2558 server.go:1019] "Starting client certificate rotation"
Sep 17 23:06:17.730048 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.724916    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 23:06:17.730048 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.724945    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 23:06:17.730048 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.726961    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 23:06:17.730048 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.728030    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 23:06:17.737022 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.736961    2558 log.go:25] "Validated CRI v1 runtime API"
Sep 17 23:06:17.742918 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.742901    2558 log.go:25] "Validated CRI v1 image API"
Sep 17 23:06:17.743708 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.743691    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 23:06:17.745789 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.745767    2558 fs.go:135] Filesystem UUIDs: map[1270e584-0ee0-4917-b09c-ecfe9bda56f9:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 986b67e7-bcc6-4155-9938-4246cc3bc656:/dev/nvme0n1p4]
Sep 17 23:06:17.745872 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.745786    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 23:06:17.749557 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.749431    2558 manager.go:216] Machine: {Timestamp:2026-09-17 23:06:17.748950737 +0000 UTC m=+1.929589165 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099043 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21140acaee31d8c6e5c8ac278a8948 SystemUUID:ec21140a-caee-31d8-c6e5-c8ac278a8948 BootID:259f78ae-c677-4722-a848-3239e3f669b7 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c0:53:98:8f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c0:53:98:8f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:4a:a0:ad:3f:81 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 23:06:17.749557 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.749547    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 23:06:17.749695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.749615    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 23:06:17.751252 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.751227    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 23:06:17.751377 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.751253    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-236.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 23:06:17.752564 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.752554    2558 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 23:06:17.752606 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.752566    2558 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 23:06:17.752606 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.752574    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 23:06:17.752606 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.752583    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 23:06:17.752684 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.752649    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 23:06:17.756279 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.756268    2558 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 23:06:17.756382 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.756373    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 23:06:17.756425 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.756405    2558 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 23:06:17.756425 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.756414    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 23:06:17.756425 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.756424    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 23:06:17.756557 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.756433    2558 kubelet.go:398] "Adding apiserver pod source"
Sep 17 23:06:17.756557 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.756443    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 23:06:17.756766 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.756755    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 23:06:17.756795 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.756773    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 23:06:17.758824 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.758806    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 23:06:17.763033 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.763014    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 17 23:06:17.764925 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.764909    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 23:06:17.765013 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.764939    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 23:06:17.766044 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766032    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 23:06:17.766087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766048    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 23:06:17.766087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766055    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 23:06:17.766087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766061    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 23:06:17.766087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766069    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 23:06:17.766087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766075    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 23:06:17.766087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766089    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 23:06:17.766237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766094    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 23:06:17.766237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766101    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 23:06:17.766237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766106    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 23:06:17.766237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766114    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 23:06:17.766237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766119    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 23:06:17.766237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766141    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 23:06:17.766237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.766147    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 23:06:17.767265 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:17.767241    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 23:06:17.767265 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:17.767241    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 23:06:17.770054 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.770040    2558 server.go:1268] "Started kubelet"
Sep 17 23:06:17.770132 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.770099    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 23:06:17.770695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.770656    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 23:06:17.770787 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.770704    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 23:06:17.770733 ip-10-0-137-236 systemd[1]: Started Kubernetes Kubelet.
Sep 17 23:06:17.771838 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.771615    2558 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 23:06:17.771838 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.771721    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 23:06:17.774238 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:17.774216    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 23:06:17.775341 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.775328    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-236.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 23:06:17.776504 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.776491    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 23:06:17.776812 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.776791    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 23:06:17.798048 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:17.798024    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:17.798353 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.798323    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n44lr"
Sep 17 23:06:17.798353 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.798344    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n44lr"
Sep 17 23:06:17.798569 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.798549    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 23:06:17.798569 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.798571    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 23:06:17.798830 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.798785    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 23:06:17.798910 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.798897    2558 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 23:06:17.798910 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.798909    2558 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 23:06:17.799382 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.799363    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 23:06:17.799382 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.799382    2558 factory.go:55] Registering systemd factory
Sep 17 23:06:17.799524 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.799391    2558 factory.go:223] Registration of the systemd container factory successfully
Sep 17 23:06:17.799838 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.799824    2558 factory.go:153] Registering CRI-O factory
Sep 17 23:06:17.799838 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.799839    2558 factory.go:223] Registration of the crio container factory successfully
Sep 17 23:06:17.799989 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.799855    2558 factory.go:103] Registering Raw factory
Sep 17 23:06:17.799989 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.799868    2558 manager.go:1196] Started watching for new ooms in manager
Sep 17 23:06:17.800937 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.800922    2558 manager.go:349] Starting recovery of all containers
Sep 17 23:06:17.808375 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.808136    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:06:17.810415 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:17.810397    2558 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-236.ec2.internal\" not found" node="ip-10-0-137-236.ec2.internal"
Sep 17 23:06:17.810479 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.810426    2558 manager.go:354] Recovery completed
Sep 17 23:06:17.814062 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.814050    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:17.815754 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.815739    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:17.815837 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.815771    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:17.815837 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.815786    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:17.816275 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.816261    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 23:06:17.816329 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.816276    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 23:06:17.816329 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.816291    2558 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 23:06:17.816329 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.816301    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:17.816948 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.816935    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:17.817035 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.816960    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:17.817035 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.816973    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:17.818289 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.818276    2558 policy_none.go:49] "None policy: Start"
Sep 17 23:06:17.818333 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.818291    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 23:06:17.818333 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.818301    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 23:06:17.820220 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.820209    2558 policy_none.go:47] "Start"
Sep 17 23:06:17.877671 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.858856    2558 manager.go:337] "Starting Device Plugin manager"
Sep 17 23:06:17.877671 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:17.858892    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 23:06:17.877671 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.858904    2558 server.go:87] "Starting device plugin registration server"
Sep 17 23:06:17.877671 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.859128    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 23:06:17.877671 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.859153    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 23:06:17.877671 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.859252    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 23:06:17.877671 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.859324    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 23:06:17.877671 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.859333    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 23:06:17.877671 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:17.859965    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 23:06:17.877671 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:17.860005    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:17.924207 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.924183    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 23:06:17.925311 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.925292    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 23:06:17.925380 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.925338    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 23:06:17.925380 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.925361    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 23:06:17.925462 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:17.925400    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 23:06:17.927254 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.927235    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:06:17.960097 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.960044    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:17.960796 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.960784    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:17.960853 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.960807    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:17.960853 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.960818    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:17.960853 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.960836    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-236.ec2.internal"
Sep 17 23:06:17.967387 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:17.967374    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-236.ec2.internal"
Sep 17 23:06:17.967435 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:17.967393    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-236.ec2.internal\": node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:17.977574 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:17.977555    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:18.025588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.025564    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-236.ec2.internal"]
Sep 17 23:06:18.025650 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.025634    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:18.027069 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.027055    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:18.027129 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.027084    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:18.027129 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.027094    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:18.028319 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.028306    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:18.028458 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.028444    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.028527 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.028476    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:18.029064 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.029050    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:18.029128 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.029079    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:18.029128 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.029089    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:18.029191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.029050    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:18.029191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.029151    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:18.029191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.029163    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:18.030537 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.030509    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.030592 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.030546    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:18.031540 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.031508    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:18.031603 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.031556    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:18.031603 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.031572    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:18.056093 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:18.056070    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-236.ec2.internal\" not found" node="ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.060202 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:18.060187    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-236.ec2.internal\" not found" node="ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.077744 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:18.077723    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:18.101305 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.101288    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f42ac923a9999e0f4dbf7b3962d6c4bc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal\" (UID: \"f42ac923a9999e0f4dbf7b3962d6c4bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.101392 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.101313    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f42ac923a9999e0f4dbf7b3962d6c4bc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal\" (UID: \"f42ac923a9999e0f4dbf7b3962d6c4bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.101392 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.101329    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/83f24f6f6691acab82204c1da59193da-config\") pod \"kube-apiserver-proxy-ip-10-0-137-236.ec2.internal\" (UID: \"83f24f6f6691acab82204c1da59193da\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.177803 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:18.177774    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:18.202141 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.202121    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f42ac923a9999e0f4dbf7b3962d6c4bc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal\" (UID: \"f42ac923a9999e0f4dbf7b3962d6c4bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.202209 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.202083    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f42ac923a9999e0f4dbf7b3962d6c4bc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal\" (UID: \"f42ac923a9999e0f4dbf7b3962d6c4bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.202209 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.202172    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f42ac923a9999e0f4dbf7b3962d6c4bc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal\" (UID: \"f42ac923a9999e0f4dbf7b3962d6c4bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.202209 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.202188    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/83f24f6f6691acab82204c1da59193da-config\") pod \"kube-apiserver-proxy-ip-10-0-137-236.ec2.internal\" (UID: \"83f24f6f6691acab82204c1da59193da\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.202313 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.202219    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/83f24f6f6691acab82204c1da59193da-config\") pod \"kube-apiserver-proxy-ip-10-0-137-236.ec2.internal\" (UID: \"83f24f6f6691acab82204c1da59193da\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.202313 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.202247    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f42ac923a9999e0f4dbf7b3962d6c4bc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal\" (UID: \"f42ac923a9999e0f4dbf7b3962d6c4bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.278510 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:18.278468    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:18.358976 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.358960    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.364236 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.364222    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:18.379544 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:18.379523    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:18.480072 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:18.480042    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:18.580613 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:18.580560    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:18.681224 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:18.681198    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:18.724731 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.724711    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 23:06:18.725233 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.724878    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 23:06:18.725233 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.724888    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 23:06:18.781922 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:18.781894    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:18.804718 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.804686    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 23:01:17 +0000 UTC" deadline="2028-03-22 21:32:34.869492923 +0000 UTC"
Sep 17 23:06:18.804718 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:18.804711    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13246h26m16.064784863s"
Sep 17 23:06:18.882092 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:18.882072    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:18.982631 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:18.982614    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:19.011594 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:19.011567    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf42ac923a9999e0f4dbf7b3962d6c4bc.slice/crio-cbc2c9884aa0882eb0dec7f839f572261b2b22df8893838eb0fe63a5dc04010f WatchSource:0}: Error finding container cbc2c9884aa0882eb0dec7f839f572261b2b22df8893838eb0fe63a5dc04010f: Status 404 returned error can't find the container with id cbc2c9884aa0882eb0dec7f839f572261b2b22df8893838eb0fe63a5dc04010f
Sep 17 23:06:19.012166 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:19.012148    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83f24f6f6691acab82204c1da59193da.slice/crio-6479295658a6015e6bd7b9cf74940fb5518422666257e3ac127cbcd595eb1bef WatchSource:0}: Error finding container 6479295658a6015e6bd7b9cf74940fb5518422666257e3ac127cbcd595eb1bef: Status 404 returned error can't find the container with id 6479295658a6015e6bd7b9cf74940fb5518422666257e3ac127cbcd595eb1bef
Sep 17 23:06:19.018151 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.018135    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:06:19.083381 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:19.083358    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:19.100147 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.100130    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:06:19.184426 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:19.184390    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-236.ec2.internal\" not found"
Sep 17 23:06:19.211587 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.211566    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:06:19.298720 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.298699    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:19.308549 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.308534    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 23:06:19.309595 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.309581    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-236.ec2.internal"
Sep 17 23:06:19.314988 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.314974    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 23:06:19.617091 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.617067    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:06:19.730243 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.730220    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:06:19.758037 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.758016    2558 apiserver.go:52] "Watching apiserver"
Sep 17 23:06:19.763668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.763644    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 23:06:19.764056 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.764034    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-ngfq2","openshift-image-registry/node-ca-cxgg6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal","openshift-multus/multus-rvbth","openshift-multus/network-metrics-daemon-f4lpx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j","openshift-dns/node-resolver-nmb7f","openshift-multus/multus-additional-cni-plugins-zrtzv","openshift-network-diagnostics/network-check-target-7bck7","openshift-network-operator/iptables-alerter-gwbhj","openshift-ovn-kubernetes/ovnkube-node-lhs72","kube-system/global-pull-secret-syncer-6tvk2","kube-system/konnectivity-agent-hbpfg","kube-system/kube-apiserver-proxy-ip-10-0-137-236.ec2.internal"]
Sep 17 23:06:19.766636 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.766617    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gwbhj"
Sep 17 23:06:19.768726 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.768706    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ltsbr\""
Sep 17 23:06:19.768933 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.768914    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 23:06:19.769027 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.769004    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 23:06:19.769176 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.769161    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 23:06:19.769471 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.769454    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hbpfg"
Sep 17 23:06:19.772189 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.771685    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-m7zl9\""
Sep 17 23:06:19.772189 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.771870    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 23:06:19.772189 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.772053    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 23:06:19.772937 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.772535    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nmb7f"
Sep 17 23:06:19.774150 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.774132    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:19.774239 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:19.774214    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:19.774451 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.774432    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 23:06:19.775127 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.774961    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 23:06:19.775127 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.775103    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-czpdx\""
Sep 17 23:06:19.776430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.776066    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.779087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.778144    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 23:06:19.779087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.778264    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 23:06:19.779087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.778401    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 23:06:19.779087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.778500    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.779087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.778767    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-blzvj\""
Sep 17 23:06:19.779087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.778885    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 23:06:19.780506 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.780489    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:19.780605 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:19.780565    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:19.781506 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.780790    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v4xbp\""
Sep 17 23:06:19.781506 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.780875    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 23:06:19.781506 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.781046    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 23:06:19.781506 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.781070    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 23:06:19.783808 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.783792    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cxgg6"
Sep 17 23:06:19.786176 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.786138    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.786861 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.786377    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c2mkq\""
Sep 17 23:06:19.786861 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.786482    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 23:06:19.786861 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.786141    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 23:06:19.786861 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.786616    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 23:06:19.788665 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.788646    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 23:06:19.788994 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.788974    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sjvpx\""
Sep 17 23:06:19.789226 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.789209    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 23:06:19.789460 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.789442    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.791599 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.791584    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 23:06:19.792369 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.791829    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 23:06:19.792369 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.791659    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 23:06:19.792369 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.792050    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 23:06:19.792369 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.792085    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 23:06:19.792369 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.792118    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vqvrf\""
Sep 17 23:06:19.792369 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.792226    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 23:06:19.792369 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.792350    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:19.792760 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:19.792421    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:19.793897 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.793879    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.795917 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.795874    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 23:06:19.796378 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.796274    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bht7f\""
Sep 17 23:06:19.796378 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.796374    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 23:06:19.799574 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.798947    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 23:06:19.800177 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.800160    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 23:06:19.808844 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.808822    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-multus-socket-dir-parent\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.808936 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.808868    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-run-ovn-kubernetes\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.808936 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.808893    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e34c616-c82c-4618-a991-5fb36612f2df-tmp-dir\") pod \"node-resolver-nmb7f\" (UID: \"0e34c616-c82c-4618-a991-5fb36612f2df\") " pod="openshift-dns/node-resolver-nmb7f"
Sep 17 23:06:19.808936 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.808916    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-sys\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.809091 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.808939    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.809091 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.808962    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-var-lib-openvswitch\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.809091 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809058    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmpxt\" (UniqueName: \"kubernetes.io/projected/569bb01a-b894-4327-843d-9ed9959567c0-kube-api-access-hmpxt\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:19.809234 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809084    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-cni-binary-copy\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.809234 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809129    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2bfb\" (UniqueName: \"kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb\") pod \"network-check-target-7bck7\" (UID: \"03b93316-5332-483d-87ea-fc94869caa29\") " pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:19.809234 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809170    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-run-openvswitch\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.809234 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809194    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-cni-netd\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.809473 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.809473 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809264    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-sysconfig\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.809473 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809286    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-kubernetes\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.809473 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809308    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-var-lib-kubelet\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.809473 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809339    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-var-lib-cni-bin\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.809473 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809362    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-slash\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.809473 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809383    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-etc-openvswitch\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.809473 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809438    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-cnibin\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809485    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809530    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-systemd-units\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809564    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809591    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c9a28b22-04f6-4e93-a852-2e3cf1ce146f-agent-certs\") pod \"konnectivity-agent-hbpfg\" (UID: \"c9a28b22-04f6-4e93-a852-2e3cf1ce146f\") " pod="kube-system/konnectivity-agent-hbpfg"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809631    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-run\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809654    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-cni-binary-copy\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809678    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-multus-cni-dir\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809700    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-run-k8s-cni-cncf-io\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809721    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-lib-modules\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809743    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809765    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v2j2\" (UniqueName: \"kubernetes.io/projected/6782274a-ac3d-41e1-ad5d-d221b7abb981-kube-api-access-5v2j2\") pod \"node-ca-cxgg6\" (UID: \"6782274a-ac3d-41e1-ad5d-d221b7abb981\") " pod="openshift-image-registry/node-ca-cxgg6"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809796    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-hostroot\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809819    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-etc-kubernetes\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809842    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5e04b2c-70dc-4136-802d-d56a06817944-env-overrides\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.809907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809877    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5e04b2c-70dc-4136-802d-d56a06817944-ovn-node-metrics-cert\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809900    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km24p\" (UniqueName: \"kubernetes.io/projected/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-kube-api-access-km24p\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.809971    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-system-cni-dir\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810009    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-var-lib-cni-multus\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810034    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-multus-conf-dir\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810059    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-multus-daemon-config\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810082    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-run-systemd\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810106    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-run-ovn\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810127    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-systemd\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810153    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqhmb\" (UniqueName: \"kubernetes.io/projected/0e34c616-c82c-4618-a991-5fb36612f2df-kube-api-access-zqhmb\") pod \"node-resolver-nmb7f\" (UID: \"0e34c616-c82c-4618-a991-5fb36612f2df\") " pod="openshift-dns/node-resolver-nmb7f"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810174    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810195    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-device-dir\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810223    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc5fv\" (UniqueName: \"kubernetes.io/projected/f34c3529-2cce-4809-a370-51e977b71aba-kube-api-access-zc5fv\") pod \"iptables-alerter-gwbhj\" (UID: \"f34c3529-2cce-4809-a370-51e977b71aba\") " pod="openshift-network-operator/iptables-alerter-gwbhj"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810237    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c9a28b22-04f6-4e93-a852-2e3cf1ce146f-konnectivity-ca\") pod \"konnectivity-agent-hbpfg\" (UID: \"c9a28b22-04f6-4e93-a852-2e3cf1ce146f\") " pod="kube-system/konnectivity-agent-hbpfg"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810251    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0e34c616-c82c-4618-a991-5fb36612f2df-hosts-file\") pod \"node-resolver-nmb7f\" (UID: \"0e34c616-c82c-4618-a991-5fb36612f2df\") " pod="openshift-dns/node-resolver-nmb7f"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810267    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-socket-dir\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810281    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-etc-selinux\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810296    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-sys-fs\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810310    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52ffc087-d6a2-4e15-85a7-2b6901f312b4-tmp\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810345    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f34c3529-2cce-4809-a370-51e977b71aba-iptables-alerter-script\") pod \"iptables-alerter-gwbhj\" (UID: \"f34c3529-2cce-4809-a370-51e977b71aba\") " pod="openshift-network-operator/iptables-alerter-gwbhj"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810491    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f34c3529-2cce-4809-a370-51e977b71aba-host-slash\") pod \"iptables-alerter-gwbhj\" (UID: \"f34c3529-2cce-4809-a370-51e977b71aba\") " pod="openshift-network-operator/iptables-alerter-gwbhj"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810550    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-run-netns\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810590    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z78g6\" (UniqueName: \"kubernetes.io/projected/e5e04b2c-70dc-4136-802d-d56a06817944-kube-api-access-z78g6\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810669    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-sysctl-conf\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810707    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-host\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810742    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-system-cni-dir\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810766    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6782274a-ac3d-41e1-ad5d-d221b7abb981-serviceca\") pod \"node-ca-cxgg6\" (UID: \"6782274a-ac3d-41e1-ad5d-d221b7abb981\") " pod="openshift-image-registry/node-ca-cxgg6"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810788    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-log-socket\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810810    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5qxk\" (UniqueName: \"kubernetes.io/projected/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-kube-api-access-q5qxk\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810835    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810843    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-kubelet\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810866    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-node-log\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810889    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5e04b2c-70dc-4136-802d-d56a06817944-ovnkube-config\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.811430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810912    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-registration-dir\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810935    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fzj6\" (UniqueName: \"kubernetes.io/projected/c9a16693-24b5-4131-b971-92be96805852-kube-api-access-2fzj6\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810958    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-modprobe-d\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.810998    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-tuned\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.811024    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-var-lib-kubelet\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.811067    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-cni-bin\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.811092    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-dbus\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.811131    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-run-netns\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.811155    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-kubelet-config\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.811192    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d95mr\" (UniqueName: \"kubernetes.io/projected/52ffc087-d6a2-4e15-85a7-2b6901f312b4-kube-api-access-d95mr\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.811233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-cnibin\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.811267    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-run-multus-certs\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.811285    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6782274a-ac3d-41e1-ad5d-d221b7abb981-host\") pod \"node-ca-cxgg6\" (UID: \"6782274a-ac3d-41e1-ad5d-d221b7abb981\") " pod="openshift-image-registry/node-ca-cxgg6"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.811323    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5e04b2c-70dc-4136-802d-d56a06817944-ovnkube-script-lib\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.811345    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-sysctl-d\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.812200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.811361    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.836589 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.836562    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ptwpq"
Sep 17 23:06:19.855333 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.855306    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ptwpq"
Sep 17 23:06:19.912394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912334    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-run-netns\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.912394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912366    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z78g6\" (UniqueName: \"kubernetes.io/projected/e5e04b2c-70dc-4136-802d-d56a06817944-kube-api-access-z78g6\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.912394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912380    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-sysctl-conf\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.912394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912394    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-host\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912408    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-system-cni-dir\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912428    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6782274a-ac3d-41e1-ad5d-d221b7abb981-serviceca\") pod \"node-ca-cxgg6\" (UID: \"6782274a-ac3d-41e1-ad5d-d221b7abb981\") " pod="openshift-image-registry/node-ca-cxgg6"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912449    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-log-socket\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912470    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5qxk\" (UniqueName: \"kubernetes.io/projected/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-kube-api-access-q5qxk\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912486    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-kubelet\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912500    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-node-log\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912534    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5e04b2c-70dc-4136-802d-d56a06817944-ovnkube-config\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912548    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-registration-dir\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912563    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fzj6\" (UniqueName: \"kubernetes.io/projected/c9a16693-24b5-4131-b971-92be96805852-kube-api-access-2fzj6\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912578    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-modprobe-d\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912597    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-tuned\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912611    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-var-lib-kubelet\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912625    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-cni-bin\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912639    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-dbus\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912653    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-run-netns\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912673    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-kubelet-config\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:19.912691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912689    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d95mr\" (UniqueName: \"kubernetes.io/projected/52ffc087-d6a2-4e15-85a7-2b6901f312b4-kube-api-access-d95mr\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912704    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-cnibin\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912727    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-run-multus-certs\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912748    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6782274a-ac3d-41e1-ad5d-d221b7abb981-host\") pod \"node-ca-cxgg6\" (UID: \"6782274a-ac3d-41e1-ad5d-d221b7abb981\") " pod="openshift-image-registry/node-ca-cxgg6"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912771    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5e04b2c-70dc-4136-802d-d56a06817944-ovnkube-script-lib\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912787    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-sysctl-d\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912810    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912831    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-multus-socket-dir-parent\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912845    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-run-ovn-kubernetes\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912859    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e34c616-c82c-4618-a991-5fb36612f2df-tmp-dir\") pod \"node-resolver-nmb7f\" (UID: \"0e34c616-c82c-4618-a991-5fb36612f2df\") " pod="openshift-dns/node-resolver-nmb7f"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912878    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-sys\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912899    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912924    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-var-lib-openvswitch\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912947    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmpxt\" (UniqueName: \"kubernetes.io/projected/569bb01a-b894-4327-843d-9ed9959567c0-kube-api-access-hmpxt\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912971    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-cni-binary-copy\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.912995    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bfb\" (UniqueName: \"kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb\") pod \"network-check-target-7bck7\" (UID: \"03b93316-5332-483d-87ea-fc94869caa29\") " pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913019    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-run-openvswitch\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.913489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913042    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-cni-netd\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913119    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913147    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-sysconfig\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913170    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-kubernetes\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913193    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-var-lib-kubelet\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913216    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-var-lib-cni-bin\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913239    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-slash\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913262    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-etc-openvswitch\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913287    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-cnibin\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913312    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913339    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-systemd-units\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913363    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c9a28b22-04f6-4e93-a852-2e3cf1ce146f-agent-certs\") pod \"konnectivity-agent-hbpfg\" (UID: \"c9a28b22-04f6-4e93-a852-2e3cf1ce146f\") " pod="kube-system/konnectivity-agent-hbpfg"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913410    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-run\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913434    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-cni-binary-copy\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913459    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-multus-cni-dir\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913485    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-run-k8s-cni-cncf-io\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.914281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913506    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-lib-modules\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913543    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913559    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v2j2\" (UniqueName: \"kubernetes.io/projected/6782274a-ac3d-41e1-ad5d-d221b7abb981-kube-api-access-5v2j2\") pod \"node-ca-cxgg6\" (UID: \"6782274a-ac3d-41e1-ad5d-d221b7abb981\") " pod="openshift-image-registry/node-ca-cxgg6"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913574    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-hostroot\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913588    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-etc-kubernetes\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913607    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5e04b2c-70dc-4136-802d-d56a06817944-env-overrides\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913626    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5e04b2c-70dc-4136-802d-d56a06817944-ovn-node-metrics-cert\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913644    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km24p\" (UniqueName: \"kubernetes.io/projected/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-kube-api-access-km24p\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913664    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-system-cni-dir\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913681    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-var-lib-cni-multus\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913703    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-multus-conf-dir\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913731    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-multus-daemon-config\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913755    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-run-systemd\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913781    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-run-ovn\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913819    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-systemd\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913847    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqhmb\" (UniqueName: \"kubernetes.io/projected/0e34c616-c82c-4618-a991-5fb36612f2df-kube-api-access-zqhmb\") pod \"node-resolver-nmb7f\" (UID: \"0e34c616-c82c-4618-a991-5fb36612f2df\") " pod="openshift-dns/node-resolver-nmb7f"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913854    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.915040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913870    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913898    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-device-dir\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913914    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-run-netns\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913928    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc5fv\" (UniqueName: \"kubernetes.io/projected/f34c3529-2cce-4809-a370-51e977b71aba-kube-api-access-zc5fv\") pod \"iptables-alerter-gwbhj\" (UID: \"f34c3529-2cce-4809-a370-51e977b71aba\") " pod="openshift-network-operator/iptables-alerter-gwbhj"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913939    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-var-lib-kubelet\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913955    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-cni-bin\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913955    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c9a28b22-04f6-4e93-a852-2e3cf1ce146f-konnectivity-ca\") pod \"konnectivity-agent-hbpfg\" (UID: \"c9a28b22-04f6-4e93-a852-2e3cf1ce146f\") " pod="kube-system/konnectivity-agent-hbpfg"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.913999    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0e34c616-c82c-4618-a991-5fb36612f2df-hosts-file\") pod \"node-resolver-nmb7f\" (UID: \"0e34c616-c82c-4618-a991-5fb36612f2df\") " pod="openshift-dns/node-resolver-nmb7f"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914027    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-var-lib-openvswitch\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914030    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-socket-dir\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914066    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-etc-selinux\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914117    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-socket-dir\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914124    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-sys-fs\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914157    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52ffc087-d6a2-4e15-85a7-2b6901f312b4-tmp\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914185    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f34c3529-2cce-4809-a370-51e977b71aba-iptables-alerter-script\") pod \"iptables-alerter-gwbhj\" (UID: \"f34c3529-2cce-4809-a370-51e977b71aba\") " pod="openshift-network-operator/iptables-alerter-gwbhj"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914212    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f34c3529-2cce-4809-a370-51e977b71aba-host-slash\") pod \"iptables-alerter-gwbhj\" (UID: \"f34c3529-2cce-4809-a370-51e977b71aba\") " pod="openshift-network-operator/iptables-alerter-gwbhj"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914291    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f34c3529-2cce-4809-a370-51e977b71aba-host-slash\") pod \"iptables-alerter-gwbhj\" (UID: \"f34c3529-2cce-4809-a370-51e977b71aba\") " pod="openshift-network-operator/iptables-alerter-gwbhj"
Sep 17 23:06:19.915817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914299    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-sysctl-conf\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914337    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-run-netns\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914386    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-kubelet-config\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914458    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-node-log\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914482    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-dbus\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914538    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-system-cni-dir\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914548    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-host\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914166    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-sys-fs\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914593    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-log-socket\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914648    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-kubelet\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914648    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-cnibin\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914683    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-run-multus-certs\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914727    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6782274a-ac3d-41e1-ad5d-d221b7abb981-host\") pod \"node-ca-cxgg6\" (UID: \"6782274a-ac3d-41e1-ad5d-d221b7abb981\") " pod="openshift-image-registry/node-ca-cxgg6"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914760    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-modprobe-d\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.914792    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-cni-netd\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.915050    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-run-openvswitch\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.915181    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.915452    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5e04b2c-70dc-4136-802d-d56a06817944-ovnkube-config\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.916588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.915586    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-sysctl-d\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.915638    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5e04b2c-70dc-4136-802d-d56a06817944-ovnkube-script-lib\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.915664    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6782274a-ac3d-41e1-ad5d-d221b7abb981-serviceca\") pod \"node-ca-cxgg6\" (UID: \"6782274a-ac3d-41e1-ad5d-d221b7abb981\") " pod="openshift-image-registry/node-ca-cxgg6"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.915751    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-registration-dir\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.915751    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-var-lib-cni-bin\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.915823    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-sysconfig\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.915833    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-var-lib-kubelet\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.915859    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-hostroot\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.916139    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-etc-openvswitch\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.916163    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-system-cni-dir\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.916165    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5e04b2c-70dc-4136-802d-d56a06817944-env-overrides\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.916206    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-cnibin\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.916255    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-var-lib-cni-multus\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.916297    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-run-systemd\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.916345    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-multus-conf-dir\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.916402    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-run\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.916470    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-device-dir\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.916778    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 23:06:19.917358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.916879    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c9a28b22-04f6-4e93-a852-2e3cf1ce146f-konnectivity-ca\") pod \"konnectivity-agent-hbpfg\" (UID: \"c9a28b22-04f6-4e93-a852-2e3cf1ce146f\") " pod="kube-system/konnectivity-agent-hbpfg"
Sep 17 23:06:19.918163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.916887    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-multus-daemon-config\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.918163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.917147    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-run-ovn\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.918163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.917838    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-cni-binary-copy\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.918163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.917907    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-systemd\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.918163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.917983    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-multus-cni-dir\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.918628 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.918593    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-host-run-k8s-cni-cncf-io\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.918694 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.918653    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-kubernetes\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.918794 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.918778    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-lib-modules\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.919482 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:19.918778    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:19.919482 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:19.918925    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs podName:569bb01a-b894-4327-843d-9ed9959567c0 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:20.418884203 +0000 UTC m=+4.599522636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs") pod "network-metrics-daemon-f4lpx" (UID: "569bb01a-b894-4327-843d-9ed9959567c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:19.919482 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.919164    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-multus-socket-dir-parent\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.919482 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.919217    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-run-ovn-kubernetes\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.919820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.919545    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.919820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.919642    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.919820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.919670    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e34c616-c82c-4618-a991-5fb36612f2df-tmp-dir\") pod \"node-resolver-nmb7f\" (UID: \"0e34c616-c82c-4618-a991-5fb36612f2df\") " pod="openshift-dns/node-resolver-nmb7f"
Sep 17 23:06:19.919820 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:19.919719    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:19.919820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.919730    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-host-slash\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.919820 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:19.919785    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret podName:a607de1c-e68e-4985-a7d6-0c1a27f4de8b nodeName:}" failed. No retries permitted until 2026-09-17 23:06:20.419761409 +0000 UTC m=+4.600399838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret") pod "global-pull-secret-syncer-6tvk2" (UID: "a607de1c-e68e-4985-a7d6-0c1a27f4de8b") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:19.919820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.919646    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c9a16693-24b5-4131-b971-92be96805852-etc-selinux\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.920280 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.919786    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0e34c616-c82c-4618-a991-5fb36612f2df-hosts-file\") pod \"node-resolver-nmb7f\" (UID: \"0e34c616-c82c-4618-a991-5fb36612f2df\") " pod="openshift-dns/node-resolver-nmb7f"
Sep 17 23:06:19.920280 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.919859    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-etc-kubernetes\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.920280 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.919909    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5e04b2c-70dc-4136-802d-d56a06817944-systemd-units\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.920280 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.920087    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52ffc087-d6a2-4e15-85a7-2b6901f312b4-sys\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.920625 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.920315    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.920625 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.920417    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f34c3529-2cce-4809-a370-51e977b71aba-iptables-alerter-script\") pod \"iptables-alerter-gwbhj\" (UID: \"f34c3529-2cce-4809-a370-51e977b71aba\") " pod="openshift-network-operator/iptables-alerter-gwbhj"
Sep 17 23:06:19.921441 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.921302    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-cni-binary-copy\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.921793 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.921772    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5e04b2c-70dc-4136-802d-d56a06817944-ovn-node-metrics-cert\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.922580 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.922539    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c9a28b22-04f6-4e93-a852-2e3cf1ce146f-agent-certs\") pod \"konnectivity-agent-hbpfg\" (UID: \"c9a28b22-04f6-4e93-a852-2e3cf1ce146f\") " pod="kube-system/konnectivity-agent-hbpfg"
Sep 17 23:06:19.922871 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.922855    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52ffc087-d6a2-4e15-85a7-2b6901f312b4-tmp\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.923537 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.923499    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/52ffc087-d6a2-4e15-85a7-2b6901f312b4-etc-tuned\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.927543 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:19.927507    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:19.927543 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:19.927542    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:19.927693 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:19.927555    2558 projected.go:196] Error preparing data for projected volume kube-api-access-b2bfb for pod openshift-network-diagnostics/network-check-target-7bck7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:19.927693 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:19.927617    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb podName:03b93316-5332-483d-87ea-fc94869caa29 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:20.427599251 +0000 UTC m=+4.608237668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bfb" (UniqueName: "kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb") pod "network-check-target-7bck7" (UID: "03b93316-5332-483d-87ea-fc94869caa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:19.930121 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.930098    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5qxk\" (UniqueName: \"kubernetes.io/projected/9dbcbe63-94ef-4161-ba58-7a4a4349bd30-kube-api-access-q5qxk\") pod \"multus-rvbth\" (UID: \"9dbcbe63-94ef-4161-ba58-7a4a4349bd30\") " pod="openshift-multus/multus-rvbth"
Sep 17 23:06:19.930646 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.930595    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fzj6\" (UniqueName: \"kubernetes.io/projected/c9a16693-24b5-4131-b971-92be96805852-kube-api-access-2fzj6\") pod \"aws-ebs-csi-driver-node-rpd4j\" (UID: \"c9a16693-24b5-4131-b971-92be96805852\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:19.931234 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.931212    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmpxt\" (UniqueName: \"kubernetes.io/projected/569bb01a-b894-4327-843d-9ed9959567c0-kube-api-access-hmpxt\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:19.931399 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.931378    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d95mr\" (UniqueName: \"kubernetes.io/projected/52ffc087-d6a2-4e15-85a7-2b6901f312b4-kube-api-access-d95mr\") pod \"tuned-ngfq2\" (UID: \"52ffc087-d6a2-4e15-85a7-2b6901f312b4\") " pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:19.931868 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.931793    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-236.ec2.internal" event={"ID":"83f24f6f6691acab82204c1da59193da","Type":"ContainerStarted","Data":"6479295658a6015e6bd7b9cf74940fb5518422666257e3ac127cbcd595eb1bef"}
Sep 17 23:06:19.932720 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.932670    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z78g6\" (UniqueName: \"kubernetes.io/projected/e5e04b2c-70dc-4136-802d-d56a06817944-kube-api-access-z78g6\") pod \"ovnkube-node-lhs72\" (UID: \"e5e04b2c-70dc-4136-802d-d56a06817944\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:19.934430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.934371    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal" event={"ID":"f42ac923a9999e0f4dbf7b3962d6c4bc","Type":"ContainerStarted","Data":"cbc2c9884aa0882eb0dec7f839f572261b2b22df8893838eb0fe63a5dc04010f"}
Sep 17 23:06:19.936191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.936147    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqhmb\" (UniqueName: \"kubernetes.io/projected/0e34c616-c82c-4618-a991-5fb36612f2df-kube-api-access-zqhmb\") pod \"node-resolver-nmb7f\" (UID: \"0e34c616-c82c-4618-a991-5fb36612f2df\") " pod="openshift-dns/node-resolver-nmb7f"
Sep 17 23:06:19.936703 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.936685    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v2j2\" (UniqueName: \"kubernetes.io/projected/6782274a-ac3d-41e1-ad5d-d221b7abb981-kube-api-access-5v2j2\") pod \"node-ca-cxgg6\" (UID: \"6782274a-ac3d-41e1-ad5d-d221b7abb981\") " pod="openshift-image-registry/node-ca-cxgg6"
Sep 17 23:06:19.937177 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.937150    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km24p\" (UniqueName: \"kubernetes.io/projected/6fdc916f-4430-45d4-8b5b-8bd8bfa42c75-kube-api-access-km24p\") pod \"multus-additional-cni-plugins-zrtzv\" (UID: \"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75\") " pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:19.938064 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:19.938036    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc5fv\" (UniqueName: \"kubernetes.io/projected/f34c3529-2cce-4809-a370-51e977b71aba-kube-api-access-zc5fv\") pod \"iptables-alerter-gwbhj\" (UID: \"f34c3529-2cce-4809-a370-51e977b71aba\") " pod="openshift-network-operator/iptables-alerter-gwbhj"
Sep 17 23:06:20.081934 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.081903    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gwbhj"
Sep 17 23:06:20.090702 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.090681    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hbpfg"
Sep 17 23:06:20.099322 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.099305    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nmb7f"
Sep 17 23:06:20.106081 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.106061    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:20.116714 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.116694    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rvbth"
Sep 17 23:06:20.124346 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.124321    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j"
Sep 17 23:06:20.131014 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.130998    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cxgg6"
Sep 17 23:06:20.143738 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.143722    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ngfq2"
Sep 17 23:06:20.145980 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.145963    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zrtzv"
Sep 17 23:06:20.519486 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.519452    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bfb\" (UniqueName: \"kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb\") pod \"network-check-target-7bck7\" (UID: \"03b93316-5332-483d-87ea-fc94869caa29\") " pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:20.519686 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.519500    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:20.519686 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.519545    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:20.519686 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:20.519648    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:20.519686 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:20.519673    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:20.519686 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:20.519674    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:20.519686 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:20.519685    2558 projected.go:196] Error preparing data for projected volume kube-api-access-b2bfb for pod openshift-network-diagnostics/network-check-target-7bck7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:20.519967 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:20.519653    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:20.519967 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:20.519728    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret podName:a607de1c-e68e-4985-a7d6-0c1a27f4de8b nodeName:}" failed. No retries permitted until 2026-09-17 23:06:21.519712097 +0000 UTC m=+5.700350540 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret") pod "global-pull-secret-syncer-6tvk2" (UID: "a607de1c-e68e-4985-a7d6-0c1a27f4de8b") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:20.519967 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:20.519757    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs podName:569bb01a-b894-4327-843d-9ed9959567c0 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:21.519738466 +0000 UTC m=+5.700376900 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs") pod "network-metrics-daemon-f4lpx" (UID: "569bb01a-b894-4327-843d-9ed9959567c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:20.519967 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:20.519782    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb podName:03b93316-5332-483d-87ea-fc94869caa29 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:21.519771416 +0000 UTC m=+5.700409847 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bfb" (UniqueName: "kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb") pod "network-check-target-7bck7" (UID: "03b93316-5332-483d-87ea-fc94869caa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:20.858247 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.858178    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 23:01:19 +0000 UTC" deadline="2028-06-13 05:25:30.623039736 +0000 UTC"
Sep 17 23:06:20.858247 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:20.858209    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15222h19m9.764834219s"
Sep 17 23:06:21.033602 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:21.033572    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6782274a_ac3d_41e1_ad5d_d221b7abb981.slice/crio-b0ded4ca3910f6e0d2dc34a63b1ff7d1b94d2ae3b922f7119f87d677bab1792b WatchSource:0}: Error finding container b0ded4ca3910f6e0d2dc34a63b1ff7d1b94d2ae3b922f7119f87d677bab1792b: Status 404 returned error can't find the container with id b0ded4ca3910f6e0d2dc34a63b1ff7d1b94d2ae3b922f7119f87d677bab1792b
Sep 17 23:06:21.034870 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:21.034842    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a16693_24b5_4131_b971_92be96805852.slice/crio-4646f96a44f5711f5b8430ccb789aae9c1aa7afc10d837e0dd74b4769f164cc9 WatchSource:0}: Error finding container 4646f96a44f5711f5b8430ccb789aae9c1aa7afc10d837e0dd74b4769f164cc9: Status 404 returned error can't find the container with id 4646f96a44f5711f5b8430ccb789aae9c1aa7afc10d837e0dd74b4769f164cc9
Sep 17 23:06:21.035733 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:21.035670    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dbcbe63_94ef_4161_ba58_7a4a4349bd30.slice/crio-ec792fc527be099947cb045cd21e82ccd773273919d765b36bf3513dce02cfa7 WatchSource:0}: Error finding container ec792fc527be099947cb045cd21e82ccd773273919d765b36bf3513dce02cfa7: Status 404 returned error can't find the container with id ec792fc527be099947cb045cd21e82ccd773273919d765b36bf3513dce02cfa7
Sep 17 23:06:21.037355 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:21.037333    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ffc087_d6a2_4e15_85a7_2b6901f312b4.slice/crio-38467578c7db9ffa13b05b1202652f5c5eaede430ef15befc0023ffc10dc8f81 WatchSource:0}: Error finding container 38467578c7db9ffa13b05b1202652f5c5eaede430ef15befc0023ffc10dc8f81: Status 404 returned error can't find the container with id 38467578c7db9ffa13b05b1202652f5c5eaede430ef15befc0023ffc10dc8f81
Sep 17 23:06:21.039280 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:21.039022    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e04b2c_70dc_4136_802d_d56a06817944.slice/crio-be0b91e60dcdbbf337b336632c540315e8fc8c51e3dfa0e5c47604d2bb47796b WatchSource:0}: Error finding container be0b91e60dcdbbf337b336632c540315e8fc8c51e3dfa0e5c47604d2bb47796b: Status 404 returned error can't find the container with id be0b91e60dcdbbf337b336632c540315e8fc8c51e3dfa0e5c47604d2bb47796b
Sep 17 23:06:21.040680 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:21.040658    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e34c616_c82c_4618_a991_5fb36612f2df.slice/crio-a731af6f6d99246f4fa5710fc0b6eb5ea0a6b70883f9e6e74513ba71fdf27479 WatchSource:0}: Error finding container a731af6f6d99246f4fa5710fc0b6eb5ea0a6b70883f9e6e74513ba71fdf27479: Status 404 returned error can't find the container with id a731af6f6d99246f4fa5710fc0b6eb5ea0a6b70883f9e6e74513ba71fdf27479
Sep 17 23:06:21.041481 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:21.041456    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a28b22_04f6_4e93_a852_2e3cf1ce146f.slice/crio-7bf46db732c6d65071e929fc40bd754136fec38a07f8e3754d2c517339d6c87d WatchSource:0}: Error finding container 7bf46db732c6d65071e929fc40bd754136fec38a07f8e3754d2c517339d6c87d: Status 404 returned error can't find the container with id 7bf46db732c6d65071e929fc40bd754136fec38a07f8e3754d2c517339d6c87d
Sep 17 23:06:21.042578 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:21.042497    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf34c3529_2cce_4809_a370_51e977b71aba.slice/crio-120ee728333b7f447da8993b52a4079ec30c32242dddb9fafc701a9c9b9b13b8 WatchSource:0}: Error finding container 120ee728333b7f447da8993b52a4079ec30c32242dddb9fafc701a9c9b9b13b8: Status 404 returned error can't find the container with id 120ee728333b7f447da8993b52a4079ec30c32242dddb9fafc701a9c9b9b13b8
Sep 17 23:06:21.526469 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.526256    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bfb\" (UniqueName: \"kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb\") pod \"network-check-target-7bck7\" (UID: \"03b93316-5332-483d-87ea-fc94869caa29\") " pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:21.526671 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.526499    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:21.526671 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:21.526507    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:21.526671 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:21.526557    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:21.526671 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:21.526567    2558 projected.go:196] Error preparing data for projected volume kube-api-access-b2bfb for pod openshift-network-diagnostics/network-check-target-7bck7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:21.526671 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:21.526627    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb podName:03b93316-5332-483d-87ea-fc94869caa29 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:23.526612505 +0000 UTC m=+7.707250920 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bfb" (UniqueName: "kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb") pod "network-check-target-7bck7" (UID: "03b93316-5332-483d-87ea-fc94869caa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:21.526671 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.526658    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:21.527021 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:21.526758    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:21.527021 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:21.526761    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:21.527021 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:21.526808    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret podName:a607de1c-e68e-4985-a7d6-0c1a27f4de8b nodeName:}" failed. No retries permitted until 2026-09-17 23:06:23.526787606 +0000 UTC m=+7.707426037 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret") pod "global-pull-secret-syncer-6tvk2" (UID: "a607de1c-e68e-4985-a7d6-0c1a27f4de8b") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:21.527021 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:21.526839    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs podName:569bb01a-b894-4327-843d-9ed9959567c0 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:23.526821129 +0000 UTC m=+7.707459568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs") pod "network-metrics-daemon-f4lpx" (UID: "569bb01a-b894-4327-843d-9ed9959567c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:21.858806 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.858694    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 23:01:19 +0000 UTC" deadline="2028-06-15 11:08:53.21324507 +0000 UTC"
Sep 17 23:06:21.858806 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.858746    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15276h2m31.35450322s"
Sep 17 23:06:21.928573 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.927789    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:21.928573 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:21.927908    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:21.928573 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.928284    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:21.928573 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:21.928370    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:21.928573 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.928434    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:21.928573 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:21.928526    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:21.941597 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.941538    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j" event={"ID":"c9a16693-24b5-4131-b971-92be96805852","Type":"ContainerStarted","Data":"4646f96a44f5711f5b8430ccb789aae9c1aa7afc10d837e0dd74b4769f164cc9"}
Sep 17 23:06:21.953676 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.953597    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cxgg6" event={"ID":"6782274a-ac3d-41e1-ad5d-d221b7abb981","Type":"ContainerStarted","Data":"b0ded4ca3910f6e0d2dc34a63b1ff7d1b94d2ae3b922f7119f87d677bab1792b"}
Sep 17 23:06:21.964971 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.964702    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrtzv" event={"ID":"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75","Type":"ContainerStarted","Data":"5a66747da555a429052d26fab85113a47803e32ad14b6d2223e9463087171de2"}
Sep 17 23:06:21.975058 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.974996    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gwbhj" event={"ID":"f34c3529-2cce-4809-a370-51e977b71aba","Type":"ContainerStarted","Data":"120ee728333b7f447da8993b52a4079ec30c32242dddb9fafc701a9c9b9b13b8"}
Sep 17 23:06:21.976874 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.976813    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hbpfg" event={"ID":"c9a28b22-04f6-4e93-a852-2e3cf1ce146f","Type":"ContainerStarted","Data":"7bf46db732c6d65071e929fc40bd754136fec38a07f8e3754d2c517339d6c87d"}
Sep 17 23:06:21.981986 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.981897    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72" event={"ID":"e5e04b2c-70dc-4136-802d-d56a06817944","Type":"ContainerStarted","Data":"be0b91e60dcdbbf337b336632c540315e8fc8c51e3dfa0e5c47604d2bb47796b"}
Sep 17 23:06:21.989623 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.989562    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rvbth" event={"ID":"9dbcbe63-94ef-4161-ba58-7a4a4349bd30","Type":"ContainerStarted","Data":"ec792fc527be099947cb045cd21e82ccd773273919d765b36bf3513dce02cfa7"}
Sep 17 23:06:21.993060 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.992495    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-236.ec2.internal" event={"ID":"83f24f6f6691acab82204c1da59193da","Type":"ContainerStarted","Data":"0617c6a905e7e583f3522a939965d74c01bb0fb05a99a3be6b4f59909171ce98"}
Sep 17 23:06:21.999483 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.998343    2558 generic.go:356] "Generic (PLEG): container finished" podID="f42ac923a9999e0f4dbf7b3962d6c4bc" containerID="eac7d35c5bc1d89768f96ebfe90464e92339e14b9afbf0f04b8113836712bf22" exitCode=0
Sep 17 23:06:21.999483 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:21.998412    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal" event={"ID":"f42ac923a9999e0f4dbf7b3962d6c4bc","Type":"ContainerDied","Data":"eac7d35c5bc1d89768f96ebfe90464e92339e14b9afbf0f04b8113836712bf22"}
Sep 17 23:06:22.002975 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:22.002950    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nmb7f" event={"ID":"0e34c616-c82c-4618-a991-5fb36612f2df","Type":"ContainerStarted","Data":"a731af6f6d99246f4fa5710fc0b6eb5ea0a6b70883f9e6e74513ba71fdf27479"}
Sep 17 23:06:22.010335 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:22.010238    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-236.ec2.internal" podStartSLOduration=3.010220974 podStartE2EDuration="3.010220974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:22.010019835 +0000 UTC m=+6.190658280" watchObservedRunningTime="2026-09-17 23:06:22.010220974 +0000 UTC m=+6.190859410"
Sep 17 23:06:22.016197 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:22.016145    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ngfq2" event={"ID":"52ffc087-d6a2-4e15-85a7-2b6901f312b4","Type":"ContainerStarted","Data":"38467578c7db9ffa13b05b1202652f5c5eaede430ef15befc0023ffc10dc8f81"}
Sep 17 23:06:23.042026 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:23.041587    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal" event={"ID":"f42ac923a9999e0f4dbf7b3962d6c4bc","Type":"ContainerStarted","Data":"2f643964211eff7cdb9b39aeee398b6d438e7a9708c9d2fd2eeee697f6e20080"}
Sep 17 23:06:23.056885 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:23.055822    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-236.ec2.internal" podStartSLOduration=4.055800895 podStartE2EDuration="4.055800895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:23.05569331 +0000 UTC m=+7.236331749" watchObservedRunningTime="2026-09-17 23:06:23.055800895 +0000 UTC m=+7.236439334"
Sep 17 23:06:23.545883 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:23.545843    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bfb\" (UniqueName: \"kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb\") pod \"network-check-target-7bck7\" (UID: \"03b93316-5332-483d-87ea-fc94869caa29\") " pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:23.546083 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:23.545907    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:23.546083 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:23.545938    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:23.546304 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:23.546090    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:23.546304 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:23.546153    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs podName:569bb01a-b894-4327-843d-9ed9959567c0 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:27.546135639 +0000 UTC m=+11.726774060 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs") pod "network-metrics-daemon-f4lpx" (UID: "569bb01a-b894-4327-843d-9ed9959567c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:23.546596 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:23.546577    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:23.546697 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:23.546611    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:23.546697 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:23.546624    2558 projected.go:196] Error preparing data for projected volume kube-api-access-b2bfb for pod openshift-network-diagnostics/network-check-target-7bck7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:23.546697 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:23.546672    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb podName:03b93316-5332-483d-87ea-fc94869caa29 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:27.546657484 +0000 UTC m=+11.727295905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bfb" (UniqueName: "kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb") pod "network-check-target-7bck7" (UID: "03b93316-5332-483d-87ea-fc94869caa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:23.546855 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:23.546734    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:23.546855 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:23.546766    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret podName:a607de1c-e68e-4985-a7d6-0c1a27f4de8b nodeName:}" failed. No retries permitted until 2026-09-17 23:06:27.54675568 +0000 UTC m=+11.727394111 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret") pod "global-pull-secret-syncer-6tvk2" (UID: "a607de1c-e68e-4985-a7d6-0c1a27f4de8b") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:23.927994 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:23.927917    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:23.928139 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:23.928045    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:23.928845 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:23.928430    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:23.928845 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:23.928556    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:23.928845 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:23.928622    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:23.928845 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:23.928745    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:25.926026 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:25.925993    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:25.926573 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:25.926145    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:25.926573 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:25.926184    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:25.926573 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:25.926260    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:25.926573 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:25.926308    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:25.926573 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:25.926372    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:27.581554 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:27.581445    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:27.581554 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:27.581549    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bfb\" (UniqueName: \"kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb\") pod \"network-check-target-7bck7\" (UID: \"03b93316-5332-483d-87ea-fc94869caa29\") " pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:27.582064 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:27.581587    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:27.582064 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:27.581703    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:27.582064 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:27.581764    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret podName:a607de1c-e68e-4985-a7d6-0c1a27f4de8b nodeName:}" failed. No retries permitted until 2026-09-17 23:06:35.581744916 +0000 UTC m=+19.762383352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret") pod "global-pull-secret-syncer-6tvk2" (UID: "a607de1c-e68e-4985-a7d6-0c1a27f4de8b") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:27.582064 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:27.581825    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:27.582064 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:27.581861    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs podName:569bb01a-b894-4327-843d-9ed9959567c0 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:35.581848636 +0000 UTC m=+19.762487067 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs") pod "network-metrics-daemon-f4lpx" (UID: "569bb01a-b894-4327-843d-9ed9959567c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:27.582064 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:27.581933    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:27.582064 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:27.581947    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:27.582064 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:27.581957    2558 projected.go:196] Error preparing data for projected volume kube-api-access-b2bfb for pod openshift-network-diagnostics/network-check-target-7bck7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:27.582064 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:27.581990    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb podName:03b93316-5332-483d-87ea-fc94869caa29 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:35.581979839 +0000 UTC m=+19.762618269 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bfb" (UniqueName: "kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb") pod "network-check-target-7bck7" (UID: "03b93316-5332-483d-87ea-fc94869caa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:27.932726 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:27.932654    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:27.932878 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:27.932772    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:27.933152 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:27.933129    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:27.933263 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:27.933218    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:27.933335 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:27.933271    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:27.933389 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:27.933340    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:29.929068 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:29.929036    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:29.929534 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:29.929151    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:29.929534 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:29.929158    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:29.929534 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:29.929274    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:29.929534 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:29.929315    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:29.929534 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:29.929393    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:31.931544 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:31.931501    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:31.932068 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:31.931501    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:31.932068 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:31.931637    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:31.932068 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:31.931525    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:31.932068 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:31.931714    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:31.932068 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:31.931794    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:33.925987 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:33.925955    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:33.926474 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:33.925964    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:33.926474 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:33.926071    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:33.926474 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:33.926080    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:33.926474 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:33.926207    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:33.926474 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:33.926317    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:35.641013 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:35.640979    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:35.641443 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:35.641027    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:35.641443 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:35.641088    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bfb\" (UniqueName: \"kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb\") pod \"network-check-target-7bck7\" (UID: \"03b93316-5332-483d-87ea-fc94869caa29\") " pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:35.641443 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:35.641125    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:35.641443 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:35.641196    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret podName:a607de1c-e68e-4985-a7d6-0c1a27f4de8b nodeName:}" failed. No retries permitted until 2026-09-17 23:06:51.641177294 +0000 UTC m=+35.821815735 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret") pod "global-pull-secret-syncer-6tvk2" (UID: "a607de1c-e68e-4985-a7d6-0c1a27f4de8b") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:35.641443 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:35.641200    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:35.641443 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:35.641206    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:35.641443 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:35.641225    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:35.641443 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:35.641238    2558 projected.go:196] Error preparing data for projected volume kube-api-access-b2bfb for pod openshift-network-diagnostics/network-check-target-7bck7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:35.641443 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:35.641252    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs podName:569bb01a-b894-4327-843d-9ed9959567c0 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:51.641239694 +0000 UTC m=+35.821878126 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs") pod "network-metrics-daemon-f4lpx" (UID: "569bb01a-b894-4327-843d-9ed9959567c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:35.641443 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:35.641280    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb podName:03b93316-5332-483d-87ea-fc94869caa29 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:51.64126738 +0000 UTC m=+35.821905801 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bfb" (UniqueName: "kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb") pod "network-check-target-7bck7" (UID: "03b93316-5332-483d-87ea-fc94869caa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:35.926263 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:35.926112    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:35.926263 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:35.926125    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:35.926263 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:35.926242    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:35.926263 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:35.926243    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:35.926625 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:35.926341    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:35.926625 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:35.926423    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:37.926595 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:37.926565    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:37.927009 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:37.926641    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:37.927009 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:37.926644    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:37.927009 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:37.926765    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:37.927009 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:37.926774    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:37.927009 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:37.926850    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:39.072931 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.072689    2558 generic.go:356] "Generic (PLEG): container finished" podID="6fdc916f-4430-45d4-8b5b-8bd8bfa42c75" containerID="f171ed7a541f95b0ec13ddc0c5c1bb6eab8444bdf62e0847302eb221f366ba67" exitCode=0
Sep 17 23:06:39.073771 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.072781    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrtzv" event={"ID":"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75","Type":"ContainerDied","Data":"f171ed7a541f95b0ec13ddc0c5c1bb6eab8444bdf62e0847302eb221f366ba67"}
Sep 17 23:06:39.075029 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.074991    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hbpfg" event={"ID":"c9a28b22-04f6-4e93-a852-2e3cf1ce146f","Type":"ContainerStarted","Data":"1a654fa871da6ca21e8dce75ffc28d03c6f26c86d00875da04236eb091c4f684"}
Sep 17 23:06:39.078483 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.078457    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72" event={"ID":"e5e04b2c-70dc-4136-802d-d56a06817944","Type":"ContainerStarted","Data":"d6de7821ee220f7a43045f477651f6dd150ca19f20a8b71dd65be1bd46cb1adf"}
Sep 17 23:06:39.078588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.078560    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72" event={"ID":"e5e04b2c-70dc-4136-802d-d56a06817944","Type":"ContainerStarted","Data":"6f18b6e8568b1b566df6f63c1532b0fc0872773860c407e3f841c5f87b5f9354"}
Sep 17 23:06:39.078623 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.078614    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72" event={"ID":"e5e04b2c-70dc-4136-802d-d56a06817944","Type":"ContainerStarted","Data":"b03706bd82e9dd694bf09b3fbc8fabc58646c32475a8543fdf2f311e37164f64"}
Sep 17 23:06:39.078675 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.078662    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72" event={"ID":"e5e04b2c-70dc-4136-802d-d56a06817944","Type":"ContainerStarted","Data":"5ab4b3d68541b29c74b755cb2beebe7b41db610d3363a993d67ac9c07e06798d"}
Sep 17 23:06:39.078726 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.078713    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72" event={"ID":"e5e04b2c-70dc-4136-802d-d56a06817944","Type":"ContainerStarted","Data":"938a18419789931327f4b0a8491b1e628d1f47fe9192fd2d606b3795f77ac911"}
Sep 17 23:06:39.078777 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.078764    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72" event={"ID":"e5e04b2c-70dc-4136-802d-d56a06817944","Type":"ContainerStarted","Data":"b3afcca361e912bd7d424ffad920d4d4903c8c81da16aa81402430a889eaa9d3"}
Sep 17 23:06:39.080189 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.080167    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rvbth" event={"ID":"9dbcbe63-94ef-4161-ba58-7a4a4349bd30","Type":"ContainerStarted","Data":"64a8ed82b0f14407a058f8e88eab4c38853e78ace7ce34060f3a9b267e11b30a"}
Sep 17 23:06:39.081647 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.081625    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nmb7f" event={"ID":"0e34c616-c82c-4618-a991-5fb36612f2df","Type":"ContainerStarted","Data":"abc777d85f81e0de1cac1a17f6133e0e72754ab646f988bc17282a9252b9ee30"}
Sep 17 23:06:39.083141 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.083120    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ngfq2" event={"ID":"52ffc087-d6a2-4e15-85a7-2b6901f312b4","Type":"ContainerStarted","Data":"6290297c859a7f6caf37f6739443c9ba5824aaeae7f922a623fe18487e596529"}
Sep 17 23:06:39.084611 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.084591    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j" event={"ID":"c9a16693-24b5-4131-b971-92be96805852","Type":"ContainerStarted","Data":"d0908d648e49a00fbffc9ed416f52f6bf9ed5f31a7c7410bde157efb15ee6ea1"}
Sep 17 23:06:39.086002 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.085985    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cxgg6" event={"ID":"6782274a-ac3d-41e1-ad5d-d221b7abb981","Type":"ContainerStarted","Data":"7d7e20d9563c47771ab1d9effa4bd8ffcb465bf9e5c218dba25aa3554d66d4b2"}
Sep 17 23:06:39.123852 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.123800    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hbpfg" podStartSLOduration=9.063890916 podStartE2EDuration="22.123787292s" totalImagesPullingTime="13.059896376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:39.109168244 +0000 UTC m=+23.289806682" watchObservedRunningTime="2026-09-17 23:06:39.123787292 +0000 UTC m=+23.304425734"
Sep 17 23:06:39.140611 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.140573    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nmb7f" podStartSLOduration=9.0800582 podStartE2EDuration="22.140562241s" totalImagesPullingTime="13.060504041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:39.123496169 +0000 UTC m=+23.304134607" watchObservedRunningTime="2026-09-17 23:06:39.140562241 +0000 UTC m=+23.321200679"
Sep 17 23:06:39.141099 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.141068    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rvbth" podStartSLOduration=3.8583051250000002 podStartE2EDuration="21.141058209s" totalImagesPullingTime="17.282753084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:39.140537078 +0000 UTC m=+23.321175516" watchObservedRunningTime="2026-09-17 23:06:39.141058209 +0000 UTC m=+23.321696647"
Sep 17 23:06:39.169837 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.169393    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ngfq2" podStartSLOduration=3.950045559 podStartE2EDuration="21.16937937s" totalImagesPullingTime="17.219333811s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:39.169338584 +0000 UTC m=+23.349977024" watchObservedRunningTime="2026-09-17 23:06:39.16937937 +0000 UTC m=+23.350017810"
Sep 17 23:06:39.169837 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.169507    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cxgg6" podStartSLOduration=3.949994933 podStartE2EDuration="21.169501835s" totalImagesPullingTime="17.219506902s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:39.153978614 +0000 UTC m=+23.334617052" watchObservedRunningTime="2026-09-17 23:06:39.169501835 +0000 UTC m=+23.350140273"
Sep 17 23:06:39.282900 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.282866    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:39.337675 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.337654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:39.345103 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.345075    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 23:06:39.872410 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.872301    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T23:06:39.345100106Z","UUID":"5b073249-43fd-4dc5-a827-897f32ff0b62","Handler":null,"Name":"","Endpoint":""}
Sep 17 23:06:39.874378 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.874359    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 23:06:39.874527 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.874391    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 23:06:39.926615 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.926541    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:39.926759 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.926555    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:39.926759 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:39.926665    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:39.926759 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.926684    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:39.926912 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:39.926762    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:39.926912 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:39.926838    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:39.998213 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:39.998176    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:40.058198 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:40.058167    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:40.090275 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:40.090247    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j" event={"ID":"c9a16693-24b5-4131-b971-92be96805852","Type":"ContainerStarted","Data":"41a278df3b94a687d929977af9e48d98c68bd355fcdb5d42daff09a6ca95a66f"}
Sep 17 23:06:40.092240 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:40.092197    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gwbhj" event={"ID":"f34c3529-2cce-4809-a370-51e977b71aba","Type":"ContainerStarted","Data":"04e02182976ca4ce5eeccc169a6ec3d89f958537f7fc94020dbb1c7da57283a6"}
Sep 17 23:06:40.106218 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:40.106177    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gwbhj" podStartSLOduration=5.895227666 podStartE2EDuration="23.106163588s" totalImagesPullingTime="17.210935922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:40.106117123 +0000 UTC m=+24.286755562" watchObservedRunningTime="2026-09-17 23:06:40.106163588 +0000 UTC m=+24.286802026"
Sep 17 23:06:40.224474 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:40.224447    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:40.324659 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:40.324629    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:41.096851 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:41.096812    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j" event={"ID":"c9a16693-24b5-4131-b971-92be96805852","Type":"ContainerStarted","Data":"ad53c4970e126cb46b08ec29736510f6366832c31388e9aa1aaecf25c56bef26"}
Sep 17 23:06:41.100900 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:41.100836    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72" event={"ID":"e5e04b2c-70dc-4136-802d-d56a06817944","Type":"ContainerStarted","Data":"ba561ff577437daee436fcb4b3b804fdd439e33b5854c8b77cd01f85ec50009f"}
Sep 17 23:06:41.113749 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:41.113710    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpd4j" podStartSLOduration=4.298271509 podStartE2EDuration="23.113695418s" totalImagesPullingTime="18.815423909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:41.112684789 +0000 UTC m=+25.293323227" watchObservedRunningTime="2026-09-17 23:06:41.113695418 +0000 UTC m=+25.294333855"
Sep 17 23:06:41.925924 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:41.925888    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:41.926099 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:41.925888    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:41.926099 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:41.926008    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:41.926219 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:41.926097    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:41.926219 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:41.925895    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:41.926295 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:41.926233    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:43.107442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:43.107239    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrtzv" event={"ID":"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75","Type":"ContainerStarted","Data":"9f9a2ba78b79fdc800539569fbc155a962828884dda8e4ffefcf26518edaa741"}
Sep 17 23:06:43.110996 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:43.110973    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72" event={"ID":"e5e04b2c-70dc-4136-802d-d56a06817944","Type":"ContainerStarted","Data":"a65f37f7b0783b28388d2b15616eb9534515c1caa2da9fc9a9f1ab00221eb59b"}
Sep 17 23:06:43.111288 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:43.111258    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:43.111378 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:43.111340    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:43.125191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:43.125172    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:43.151779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:43.151741    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72" podStartSLOduration=7.93139347 podStartE2EDuration="25.15172734s" totalImagesPullingTime="17.22033387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:43.150558847 +0000 UTC m=+27.331197285" watchObservedRunningTime="2026-09-17 23:06:43.15172734 +0000 UTC m=+27.332365777"
Sep 17 23:06:43.566938 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:43.566870    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hbpfg"
Sep 17 23:06:43.567428 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:43.567412    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hbpfg"
Sep 17 23:06:43.928567 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:43.928492    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:43.928567 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:43.928554    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:43.928567 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:43.928557    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:43.928747 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:43.928645    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:43.928747 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:43.928684    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:43.928815 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:43.928740    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:44.117703 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:44.117672    2558 generic.go:356] "Generic (PLEG): container finished" podID="6fdc916f-4430-45d4-8b5b-8bd8bfa42c75" containerID="9f9a2ba78b79fdc800539569fbc155a962828884dda8e4ffefcf26518edaa741" exitCode=0
Sep 17 23:06:44.118279 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:44.117752    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrtzv" event={"ID":"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75","Type":"ContainerDied","Data":"9f9a2ba78b79fdc800539569fbc155a962828884dda8e4ffefcf26518edaa741"}
Sep 17 23:06:44.118279 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:44.118252    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hbpfg"
Sep 17 23:06:44.118395 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:44.118302    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:44.118428 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:44.118400    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hbpfg"
Sep 17 23:06:44.133560 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:44.133542    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:06:44.195294 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:44.195273    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:45.045851 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:45.045676    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f4lpx"]
Sep 17 23:06:45.046014 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:45.045939    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:45.046081 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:45.046051    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:45.048915 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:45.048889    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6tvk2"]
Sep 17 23:06:45.049041 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:45.048991    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:45.049118 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:45.049095    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:45.049648 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:45.049619    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7bck7"]
Sep 17 23:06:45.049728 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:45.049720    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:45.049801 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:45.049784    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:46.123530 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:46.123482    2558 generic.go:356] "Generic (PLEG): container finished" podID="6fdc916f-4430-45d4-8b5b-8bd8bfa42c75" containerID="c9905f60e18fe6524d96d05094568beb72bc915123cc17fc46d9a129c342570b" exitCode=0
Sep 17 23:06:46.124068 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:46.123566    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrtzv" event={"ID":"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75","Type":"ContainerDied","Data":"c9905f60e18fe6524d96d05094568beb72bc915123cc17fc46d9a129c342570b"}
Sep 17 23:06:46.925636 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:46.925602    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:46.925636 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:46.925632    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:46.925817 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:46.925718    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:46.925817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:46.925771    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:46.925894 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:46.925869    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:46.925958 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:46.925945    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:48.129902 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:48.129698    2558 generic.go:356] "Generic (PLEG): container finished" podID="6fdc916f-4430-45d4-8b5b-8bd8bfa42c75" containerID="0ca5b887e5ddff6d6c31f093ee2750be96d0f4ac3bc4667059ea65b7bba8f604" exitCode=0
Sep 17 23:06:48.129902 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:48.129785    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrtzv" event={"ID":"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75","Type":"ContainerDied","Data":"0ca5b887e5ddff6d6c31f093ee2750be96d0f4ac3bc4667059ea65b7bba8f604"}
Sep 17 23:06:48.925967 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:48.925933    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:48.925967 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:48.925967    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:48.926214 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:48.926052    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:48.926214 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:48.926063    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:48.926214 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:48.926134    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:48.926214 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:48.926188    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:50.925917 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:50.925887    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:50.926408 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:50.925887    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:50.926408 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:50.925989    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bck7" podUID="03b93316-5332-483d-87ea-fc94869caa29"
Sep 17 23:06:50.926408 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:50.925891    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:50.926408 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:50.926073    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:06:50.926408 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:50.926185    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6tvk2" podUID="a607de1c-e68e-4985-a7d6-0c1a27f4de8b"
Sep 17 23:06:51.112998 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.112970    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-236.ec2.internal" event="NodeReady"
Sep 17 23:06:51.113163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.113098    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 23:06:51.156651 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.156622    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-t97lt"]
Sep 17 23:06:51.182974 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.182951    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wc6w2"]
Sep 17 23:06:51.183111 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.183065    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:51.185570 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.185548    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 23:06:51.185682 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.185583    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 23:06:51.185682 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.185614    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7hp2b\""
Sep 17 23:06:51.195245 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.195226    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t97lt"]
Sep 17 23:06:51.195357 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.195260    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wc6w2"]
Sep 17 23:06:51.195357 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.195330    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wc6w2"
Sep 17 23:06:51.197623 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.197604    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 23:06:51.197843 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.197818    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 23:06:51.197843 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.197836    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9ck9b\""
Sep 17 23:06:51.198007 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.197901    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 23:06:51.266813 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.266782    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwbs5\" (UniqueName: \"kubernetes.io/projected/d6c1ec66-3187-4649-9205-78492b636e30-kube-api-access-gwbs5\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:51.266940 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.266868    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:51.266997 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.266936    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d6c1ec66-3187-4649-9205-78492b636e30-config-volume\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:51.267034 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.267003    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6c1ec66-3187-4649-9205-78492b636e30-tmp-dir\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:51.327892 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.327832    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:51.367921 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.367893    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d6c1ec66-3187-4649-9205-78492b636e30-config-volume\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:51.368055 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.367939    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73900d7a-0740-48c9-9a8d-5e576c1316b3-cert\") pod \"ingress-canary-wc6w2\" (UID: \"73900d7a-0740-48c9-9a8d-5e576c1316b3\") " pod="openshift-ingress-canary/ingress-canary-wc6w2"
Sep 17 23:06:51.368055 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.367964    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk4qh\" (UniqueName: \"kubernetes.io/projected/73900d7a-0740-48c9-9a8d-5e576c1316b3-kube-api-access-fk4qh\") pod \"ingress-canary-wc6w2\" (UID: \"73900d7a-0740-48c9-9a8d-5e576c1316b3\") " pod="openshift-ingress-canary/ingress-canary-wc6w2"
Sep 17 23:06:51.368055 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.367997    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6c1ec66-3187-4649-9205-78492b636e30-tmp-dir\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:51.368221 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.368064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwbs5\" (UniqueName: \"kubernetes.io/projected/d6c1ec66-3187-4649-9205-78492b636e30-kube-api-access-gwbs5\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:51.368221 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.368116    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:51.368316 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:51.368220    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:06:51.368316 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:51.368276    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls podName:d6c1ec66-3187-4649-9205-78492b636e30 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:51.868257404 +0000 UTC m=+36.048895834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls") pod "dns-default-t97lt" (UID: "d6c1ec66-3187-4649-9205-78492b636e30") : secret "dns-default-metrics-tls" not found
Sep 17 23:06:51.368316 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.368287    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6c1ec66-3187-4649-9205-78492b636e30-tmp-dir\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:51.368465 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.368433    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d6c1ec66-3187-4649-9205-78492b636e30-config-volume\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:51.378939 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.378918    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwbs5\" (UniqueName: \"kubernetes.io/projected/d6c1ec66-3187-4649-9205-78492b636e30-kube-api-access-gwbs5\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:51.426987 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.426961    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:51.468917 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.468894    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73900d7a-0740-48c9-9a8d-5e576c1316b3-cert\") pod \"ingress-canary-wc6w2\" (UID: \"73900d7a-0740-48c9-9a8d-5e576c1316b3\") " pod="openshift-ingress-canary/ingress-canary-wc6w2"
Sep 17 23:06:51.469027 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.468926    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk4qh\" (UniqueName: \"kubernetes.io/projected/73900d7a-0740-48c9-9a8d-5e576c1316b3-kube-api-access-fk4qh\") pod \"ingress-canary-wc6w2\" (UID: \"73900d7a-0740-48c9-9a8d-5e576c1316b3\") " pod="openshift-ingress-canary/ingress-canary-wc6w2"
Sep 17 23:06:51.471508 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.471483    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73900d7a-0740-48c9-9a8d-5e576c1316b3-cert\") pod \"ingress-canary-wc6w2\" (UID: \"73900d7a-0740-48c9-9a8d-5e576c1316b3\") " pod="openshift-ingress-canary/ingress-canary-wc6w2"
Sep 17 23:06:51.477244 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.477227    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk4qh\" (UniqueName: \"kubernetes.io/projected/73900d7a-0740-48c9-9a8d-5e576c1316b3-kube-api-access-fk4qh\") pod \"ingress-canary-wc6w2\" (UID: \"73900d7a-0740-48c9-9a8d-5e576c1316b3\") " pod="openshift-ingress-canary/ingress-canary-wc6w2"
Sep 17 23:06:51.505292 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.505272    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wc6w2"
Sep 17 23:06:51.670631 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.670559    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:51.670631 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.670606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:51.670831 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:51.670638    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:51.670831 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.670666    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bfb\" (UniqueName: \"kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb\") pod \"network-check-target-7bck7\" (UID: \"03b93316-5332-483d-87ea-fc94869caa29\") " pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:51.670831 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:51.670708    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret podName:a607de1c-e68e-4985-a7d6-0c1a27f4de8b nodeName:}" failed. No retries permitted until 2026-09-17 23:07:23.670689172 +0000 UTC m=+67.851327603 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret") pod "global-pull-secret-syncer-6tvk2" (UID: "a607de1c-e68e-4985-a7d6-0c1a27f4de8b") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:51.670831 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:51.670765    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:51.670831 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:51.670780    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:51.670831 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:51.670800    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:51.670831 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:51.670809    2558 projected.go:196] Error preparing data for projected volume kube-api-access-b2bfb for pod openshift-network-diagnostics/network-check-target-7bck7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:51.670831 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:51.670826    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs podName:569bb01a-b894-4327-843d-9ed9959567c0 nodeName:}" failed. No retries permitted until 2026-09-17 23:07:23.670810406 +0000 UTC m=+67.851448840 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs") pod "network-metrics-daemon-f4lpx" (UID: "569bb01a-b894-4327-843d-9ed9959567c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:51.671157 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:51.670850    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb podName:03b93316-5332-483d-87ea-fc94869caa29 nodeName:}" failed. No retries permitted until 2026-09-17 23:07:23.67083663 +0000 UTC m=+67.851475077 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2bfb" (UniqueName: "kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb") pod "network-check-target-7bck7" (UID: "03b93316-5332-483d-87ea-fc94869caa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:51.872240 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:51.872206    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:51.872409 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:51.872352    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:06:51.872458 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:51.872411    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls podName:d6c1ec66-3187-4649-9205-78492b636e30 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:52.872396928 +0000 UTC m=+37.053035343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls") pod "dns-default-t97lt" (UID: "d6c1ec66-3187-4649-9205-78492b636e30") : secret "dns-default-metrics-tls" not found
Sep 17 23:06:52.359697 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:52.359575    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wc6w2"]
Sep 17 23:06:52.362820 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:06:52.362799    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73900d7a_0740_48c9_9a8d_5e576c1316b3.slice/crio-77d4d9f40802368b551a53bb3abf0cb7133e10195b429667e3d76fdda32dbad7 WatchSource:0}: Error finding container 77d4d9f40802368b551a53bb3abf0cb7133e10195b429667e3d76fdda32dbad7: Status 404 returned error can't find the container with id 77d4d9f40802368b551a53bb3abf0cb7133e10195b429667e3d76fdda32dbad7
Sep 17 23:06:52.880914 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:52.880890    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:52.881097 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:52.881015    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:06:52.881097 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:52.881062    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls podName:d6c1ec66-3187-4649-9205-78492b636e30 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:54.881048735 +0000 UTC m=+39.061687152 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls") pod "dns-default-t97lt" (UID: "d6c1ec66-3187-4649-9205-78492b636e30") : secret "dns-default-metrics-tls" not found
Sep 17 23:06:52.925888 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:52.925864    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:06:52.925973 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:52.925956    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:06:52.926009 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:52.925978    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:06:52.928735 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:52.928714    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 23:06:52.928885 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:52.928871    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 23:06:52.929681 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:52.929664    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-twn2q\""
Sep 17 23:06:52.929786 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:52.929681    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 23:06:52.929786 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:52.929682    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 23:06:52.929786 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:52.929724    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hrj9x\""
Sep 17 23:06:53.145317 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:53.145233    2558 generic.go:356] "Generic (PLEG): container finished" podID="6fdc916f-4430-45d4-8b5b-8bd8bfa42c75" containerID="71fc94e86d51c54d0f6c37092f1c03eea61e99aadb08498bd04c27e3beaba9d1" exitCode=0
Sep 17 23:06:53.145650 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:53.145312    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrtzv" event={"ID":"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75","Type":"ContainerDied","Data":"71fc94e86d51c54d0f6c37092f1c03eea61e99aadb08498bd04c27e3beaba9d1"}
Sep 17 23:06:53.147329 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:53.147302    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wc6w2" event={"ID":"73900d7a-0740-48c9-9a8d-5e576c1316b3","Type":"ContainerStarted","Data":"77d4d9f40802368b551a53bb3abf0cb7133e10195b429667e3d76fdda32dbad7"}
Sep 17 23:06:54.153464 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:54.153433    2558 generic.go:356] "Generic (PLEG): container finished" podID="6fdc916f-4430-45d4-8b5b-8bd8bfa42c75" containerID="f605fa1c5f0b38fa0b2c4fa4dd276b160c49bc0694c08580f0b3b95640f8f992" exitCode=0
Sep 17 23:06:54.153979 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:54.153471    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrtzv" event={"ID":"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75","Type":"ContainerDied","Data":"f605fa1c5f0b38fa0b2c4fa4dd276b160c49bc0694c08580f0b3b95640f8f992"}
Sep 17 23:06:54.895382 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:54.895335    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:54.895479 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:54.895437    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:06:54.895479 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:54.895479    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls podName:d6c1ec66-3187-4649-9205-78492b636e30 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:58.895466998 +0000 UTC m=+43.076105428 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls") pod "dns-default-t97lt" (UID: "d6c1ec66-3187-4649-9205-78492b636e30") : secret "dns-default-metrics-tls" not found
Sep 17 23:06:55.158844 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:55.158819    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zrtzv" event={"ID":"6fdc916f-4430-45d4-8b5b-8bd8bfa42c75","Type":"ContainerStarted","Data":"2acaafb401d408a475e907fe672252c282343a93cd86b55d458ce3e1c001525e"}
Sep 17 23:06:55.178508 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:55.178468    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zrtzv" podStartSLOduration=9.698048836 podStartE2EDuration="37.178452603s" totalImagesPullingTime="27.480403767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:55.177033126 +0000 UTC m=+39.357671564" watchObservedRunningTime="2026-09-17 23:06:55.178452603 +0000 UTC m=+39.359091044"
Sep 17 23:06:55.298988 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:55.298952    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:55.358445 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:55.358407    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:56.162290 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:56.162255    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wc6w2" event={"ID":"73900d7a-0740-48c9-9a8d-5e576c1316b3","Type":"ContainerStarted","Data":"389fe63af0f27d5664cc81342f06aec2c0994706485c6382603c3ad6faac10c5"}
Sep 17 23:06:56.177043 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:56.177001    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wc6w2" podStartSLOduration=2.488752056 podStartE2EDuration="5.17698735s" totalImagesPullingTime="2.688235294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:56.175882288 +0000 UTC m=+40.356520727" watchObservedRunningTime="2026-09-17 23:06:56.17698735 +0000 UTC m=+40.357625788"
Sep 17 23:06:56.516485 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:56.516332    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:56.569841 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:56.569816    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:58.922160 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:58.922127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:06:58.922496 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:58.922236    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:06:58.922496 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:06:58.922290    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls podName:d6c1ec66-3187-4649-9205-78492b636e30 nodeName:}" failed. No retries permitted until 2026-09-17 23:07:06.922278305 +0000 UTC m=+51.102916722 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls") pod "dns-default-t97lt" (UID: "d6c1ec66-3187-4649-9205-78492b636e30") : secret "dns-default-metrics-tls" not found
Sep 17 23:06:58.991379 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:58.991349    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:59.046172 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:06:59.046141    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:07:00.769405 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:00.769372    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:07:00.869919 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:00.869893    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:07:00.924709 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:00.924685    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:07:06.972963 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:06.972928    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:07:06.973320 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:07:06.973077    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:07:06.973320 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:07:06.973134    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls podName:d6c1ec66-3187-4649-9205-78492b636e30 nodeName:}" failed. No retries permitted until 2026-09-17 23:07:22.97311995 +0000 UTC m=+67.153758365 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls") pod "dns-default-t97lt" (UID: "d6c1ec66-3187-4649-9205-78492b636e30") : secret "dns-default-metrics-tls" not found
Sep 17 23:07:16.143234 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:16.143207    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhs72"
Sep 17 23:07:16.253343 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:16.253317    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:07:16.311848 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:16.311820    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:07:16.366358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:16.366336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:07:22.974498 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:22.974463    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:07:22.974891 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:07:22.974596    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:07:22.974891 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:07:22.974652    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls podName:d6c1ec66-3187-4649-9205-78492b636e30 nodeName:}" failed. No retries permitted until 2026-09-17 23:07:54.974638855 +0000 UTC m=+99.155277271 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls") pod "dns-default-t97lt" (UID: "d6c1ec66-3187-4649-9205-78492b636e30") : secret "dns-default-metrics-tls" not found
Sep 17 23:07:23.679878 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:23.679850    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:07:23.680041 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:23.679897    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bfb\" (UniqueName: \"kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb\") pod \"network-check-target-7bck7\" (UID: \"03b93316-5332-483d-87ea-fc94869caa29\") " pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:07:23.680093 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:23.680043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:07:23.682558 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:23.682529    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 23:07:23.682687 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:23.682583    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 23:07:23.682687 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:23.682599    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 23:07:23.690488 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:07:23.690472    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 17 23:07:23.690610 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:07:23.690596    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs podName:569bb01a-b894-4327-843d-9ed9959567c0 nodeName:}" failed. No retries permitted until 2026-09-17 23:08:27.690574885 +0000 UTC m=+131.871213321 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs") pod "network-metrics-daemon-f4lpx" (UID: "569bb01a-b894-4327-843d-9ed9959567c0") : secret "metrics-daemon-secret" not found
Sep 17 23:07:23.692444 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:23.692425    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 23:07:23.694092 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:23.694072    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a607de1c-e68e-4985-a7d6-0c1a27f4de8b-original-pull-secret\") pod \"global-pull-secret-syncer-6tvk2\" (UID: \"a607de1c-e68e-4985-a7d6-0c1a27f4de8b\") " pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:07:23.703503 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:23.703480    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2bfb\" (UniqueName: \"kubernetes.io/projected/03b93316-5332-483d-87ea-fc94869caa29-kube-api-access-b2bfb\") pod \"network-check-target-7bck7\" (UID: \"03b93316-5332-483d-87ea-fc94869caa29\") " pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:07:23.844725 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:23.844706    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-twn2q\""
Sep 17 23:07:23.849909 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:23.849898    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6tvk2"
Sep 17 23:07:23.853037 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:23.853024    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:07:24.150874 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:24.150845    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6tvk2"]
Sep 17 23:07:24.154693 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:07:24.154671    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda607de1c_e68e_4985_a7d6_0c1a27f4de8b.slice/crio-b09f838e340962ff1d0aa2b1ef3ff3b98f60202806366a6fe83b33ec633c9d5b WatchSource:0}: Error finding container b09f838e340962ff1d0aa2b1ef3ff3b98f60202806366a6fe83b33ec633c9d5b: Status 404 returned error can't find the container with id b09f838e340962ff1d0aa2b1ef3ff3b98f60202806366a6fe83b33ec633c9d5b
Sep 17 23:07:24.172597 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:24.172577    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7bck7"]
Sep 17 23:07:24.175848 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:07:24.175828    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b93316_5332_483d_87ea_fc94869caa29.slice/crio-b005b913d8150b142c921db7723c14a1405d3650274fa35df62fa56e4d923040 WatchSource:0}: Error finding container b005b913d8150b142c921db7723c14a1405d3650274fa35df62fa56e4d923040: Status 404 returned error can't find the container with id b005b913d8150b142c921db7723c14a1405d3650274fa35df62fa56e4d923040
Sep 17 23:07:24.224508 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:24.224481    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7bck7" event={"ID":"03b93316-5332-483d-87ea-fc94869caa29","Type":"ContainerStarted","Data":"b005b913d8150b142c921db7723c14a1405d3650274fa35df62fa56e4d923040"}
Sep 17 23:07:24.225501 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:24.225480    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6tvk2" event={"ID":"a607de1c-e68e-4985-a7d6-0c1a27f4de8b","Type":"ContainerStarted","Data":"b09f838e340962ff1d0aa2b1ef3ff3b98f60202806366a6fe83b33ec633c9d5b"}
Sep 17 23:07:28.415013 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.414981    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q"]
Sep 17 23:07:28.416857 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.416832    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q"
Sep 17 23:07:28.417968 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.417944    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"]
Sep 17 23:07:28.420069 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.420046    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 17 23:07:28.420069 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.420064    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-x5pz2\""
Sep 17 23:07:28.420224 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.420084    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 17 23:07:28.420224 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.420104    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 17 23:07:28.420224 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.420054    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 17 23:07:28.420389 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.420111    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.422574 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.422547    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 17 23:07:28.422693 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.422592    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 17 23:07:28.422818 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.422800    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 17 23:07:28.422972 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.422958    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 17 23:07:28.426378 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.426359    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q"]
Sep 17 23:07:28.433393 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.433377    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"]
Sep 17 23:07:28.507802 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.507772    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/992521db-fb70-4902-bc57-5d05455e1fe8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.507969 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.507827    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c57688c9d-6ww6q\" (UID: \"9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q"
Sep 17 23:07:28.507969 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.507856    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/992521db-fb70-4902-bc57-5d05455e1fe8-ca\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.507969 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.507886    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/992521db-fb70-4902-bc57-5d05455e1fe8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.507969 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.507911    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwz8v\" (UniqueName: \"kubernetes.io/projected/992521db-fb70-4902-bc57-5d05455e1fe8-kube-api-access-jwz8v\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.507969 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.507939    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/992521db-fb70-4902-bc57-5d05455e1fe8-hub\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.507969 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.507959    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdn84\" (UniqueName: \"kubernetes.io/projected/9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f-kube-api-access-xdn84\") pod \"managed-serviceaccount-addon-agent-5c57688c9d-6ww6q\" (UID: \"9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q"
Sep 17 23:07:28.508225 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.507993    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/992521db-fb70-4902-bc57-5d05455e1fe8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.608767 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.608733    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/992521db-fb70-4902-bc57-5d05455e1fe8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.608936 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.608779    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/992521db-fb70-4902-bc57-5d05455e1fe8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.608936 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.608821    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c57688c9d-6ww6q\" (UID: \"9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q"
Sep 17 23:07:28.608936 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.608858    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/992521db-fb70-4902-bc57-5d05455e1fe8-ca\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.608936 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.608889    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/992521db-fb70-4902-bc57-5d05455e1fe8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.608936 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.608910    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwz8v\" (UniqueName: \"kubernetes.io/projected/992521db-fb70-4902-bc57-5d05455e1fe8-kube-api-access-jwz8v\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.608936 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.608932    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/992521db-fb70-4902-bc57-5d05455e1fe8-hub\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.609214 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.608964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdn84\" (UniqueName: \"kubernetes.io/projected/9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f-kube-api-access-xdn84\") pod \"managed-serviceaccount-addon-agent-5c57688c9d-6ww6q\" (UID: \"9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q"
Sep 17 23:07:28.610170 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.610135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/992521db-fb70-4902-bc57-5d05455e1fe8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.612598 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.612478    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/992521db-fb70-4902-bc57-5d05455e1fe8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.612686 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.612626    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c57688c9d-6ww6q\" (UID: \"9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q"
Sep 17 23:07:28.612758 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.612734    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/992521db-fb70-4902-bc57-5d05455e1fe8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.612871 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.612850    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/992521db-fb70-4902-bc57-5d05455e1fe8-ca\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.612970 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.612946    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/992521db-fb70-4902-bc57-5d05455e1fe8-hub\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.618398 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.618367    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwz8v\" (UniqueName: \"kubernetes.io/projected/992521db-fb70-4902-bc57-5d05455e1fe8-kube-api-access-jwz8v\") pod \"cluster-proxy-proxy-agent-bb68dbc8-67nnw\" (UID: \"992521db-fb70-4902-bc57-5d05455e1fe8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:28.618576 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.618552    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdn84\" (UniqueName: \"kubernetes.io/projected/9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f-kube-api-access-xdn84\") pod \"managed-serviceaccount-addon-agent-5c57688c9d-6ww6q\" (UID: \"9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q"
Sep 17 23:07:28.736975 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.736945    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q"
Sep 17 23:07:28.742540 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:28.742504    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:07:29.026077 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:29.026004    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"]
Sep 17 23:07:29.028460 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:07:29.028437    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod992521db_fb70_4902_bc57_5d05455e1fe8.slice/crio-84e30d784064aa17def594c4dab5ec7a1eb961d5df37d64e89bc4b0adc3cb9d8 WatchSource:0}: Error finding container 84e30d784064aa17def594c4dab5ec7a1eb961d5df37d64e89bc4b0adc3cb9d8: Status 404 returned error can't find the container with id 84e30d784064aa17def594c4dab5ec7a1eb961d5df37d64e89bc4b0adc3cb9d8
Sep 17 23:07:29.029982 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:29.029963    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q"]
Sep 17 23:07:29.032832 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:07:29.032810    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b83e0f5_c9a7_47c4_acaa_ee8cadcb525f.slice/crio-a8a9f289ddb9c851d2374e43053f9204031855bc99bc50fd062196d717315c09 WatchSource:0}: Error finding container a8a9f289ddb9c851d2374e43053f9204031855bc99bc50fd062196d717315c09: Status 404 returned error can't find the container with id a8a9f289ddb9c851d2374e43053f9204031855bc99bc50fd062196d717315c09
Sep 17 23:07:29.240318 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:29.240291    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7bck7" event={"ID":"03b93316-5332-483d-87ea-fc94869caa29","Type":"ContainerStarted","Data":"2a33f36452807206b78b92f39e22ae39096ca566b3091c96415a695af8432f23"}
Sep 17 23:07:29.240426 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:29.240379    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:07:29.241364 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:29.241338    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw" event={"ID":"992521db-fb70-4902-bc57-5d05455e1fe8","Type":"ContainerStarted","Data":"84e30d784064aa17def594c4dab5ec7a1eb961d5df37d64e89bc4b0adc3cb9d8"}
Sep 17 23:07:29.242354 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:29.242337    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q" event={"ID":"9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f","Type":"ContainerStarted","Data":"a8a9f289ddb9c851d2374e43053f9204031855bc99bc50fd062196d717315c09"}
Sep 17 23:07:29.243594 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:29.243575    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6tvk2" event={"ID":"a607de1c-e68e-4985-a7d6-0c1a27f4de8b","Type":"ContainerStarted","Data":"cc906224e8ddf81e72f679ec24ab31f9b07e157a10e98019d75d0f23a66f9b75"}
Sep 17 23:07:29.255611 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:29.255570    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7bck7" podStartSLOduration=66.676799643 podStartE2EDuration="1m11.255557763s" totalImagesPullingTime="4.57875812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:29.255205601 +0000 UTC m=+73.435844040" watchObservedRunningTime="2026-09-17 23:07:29.255557763 +0000 UTC m=+73.436196194"
Sep 17 23:07:29.267939 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:29.267896    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6tvk2" podStartSLOduration=66.662994468 podStartE2EDuration="1m11.267886256s" totalImagesPullingTime="4.604891788s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:29.267589836 +0000 UTC m=+73.448228278" watchObservedRunningTime="2026-09-17 23:07:29.267886256 +0000 UTC m=+73.448524686"
Sep 17 23:07:33.258858 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:33.258822    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw" event={"ID":"992521db-fb70-4902-bc57-5d05455e1fe8","Type":"ContainerStarted","Data":"48401d9515a476161626f0e3a488e5160d53164c9c2ff003c9e2a09d8d332817"}
Sep 17 23:07:33.259283 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:33.258869    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw" event={"ID":"992521db-fb70-4902-bc57-5d05455e1fe8","Type":"ContainerStarted","Data":"dfdababdfc8c60c33da92311a573654093b01112f64abaae7966831f073bdb48"}
Sep 17 23:07:33.259283 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:33.258883    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw" event={"ID":"992521db-fb70-4902-bc57-5d05455e1fe8","Type":"ContainerStarted","Data":"7c26b95dab6999a54a661551283195a77b6e42b967a8896aed154d44761f3b00"}
Sep 17 23:07:33.260330 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:33.260306    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q" event={"ID":"9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f","Type":"ContainerStarted","Data":"b5d60faa7e1f6e69155ce44489faac0fbbc6192cb60c7fb97b3667da6fb8cdfa"}
Sep 17 23:07:33.277183 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:33.277143    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw" podStartSLOduration=1.622437427 podStartE2EDuration="5.277131226s" totalImagesPullingTime="3.654693799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:07:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:33.276878778 +0000 UTC m=+77.457517401" watchObservedRunningTime="2026-09-17 23:07:33.277131226 +0000 UTC m=+77.457769664"
Sep 17 23:07:33.290734 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:33.290700    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q" podStartSLOduration=1.650975995 podStartE2EDuration="5.290686668s" totalImagesPullingTime="3.639710673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:07:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:33.290151462 +0000 UTC m=+77.470789901" watchObservedRunningTime="2026-09-17 23:07:33.290686668 +0000 UTC m=+77.471325106"
Sep 17 23:07:54.975621 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:07:54.975595    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:07:54.975976 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:07:54.975720    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:07:54.975976 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:07:54.975776    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls podName:d6c1ec66-3187-4649-9205-78492b636e30 nodeName:}" failed. No retries permitted until 2026-09-17 23:08:58.975763267 +0000 UTC m=+163.156401683 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls") pod "dns-default-t97lt" (UID: "d6c1ec66-3187-4649-9205-78492b636e30") : secret "dns-default-metrics-tls" not found
Sep 17 23:08:00.249918 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:08:00.249890    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7bck7"
Sep 17 23:08:27.776974 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:08:27.776941    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:08:27.777440 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:08:27.777049    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 17 23:08:27.777440 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:08:27.777106    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs podName:569bb01a-b894-4327-843d-9ed9959567c0 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:29.777092844 +0000 UTC m=+253.957731274 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs") pod "network-metrics-daemon-f4lpx" (UID: "569bb01a-b894-4327-843d-9ed9959567c0") : secret "metrics-daemon-secret" not found
Sep 17 23:08:54.193709 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:08:54.193674    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-t97lt" podUID="d6c1ec66-3187-4649-9205-78492b636e30"
Sep 17 23:08:54.478094 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:08:54.478026    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t97lt"
Sep 17 23:08:55.935845 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:08:55.935802    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-f4lpx" podUID="569bb01a-b894-4327-843d-9ed9959567c0"
Sep 17 23:08:59.073164 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:08:59.073124    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:08:59.073559 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:08:59.073264    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 23:08:59.073559 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:08:59.073325    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls podName:d6c1ec66-3187-4649-9205-78492b636e30 nodeName:}" failed. No retries permitted until 2026-09-17 23:11:01.073311047 +0000 UTC m=+285.253949468 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls") pod "dns-default-t97lt" (UID: "d6c1ec66-3187-4649-9205-78492b636e30") : secret "dns-default-metrics-tls" not found
Sep 17 23:09:08.729669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:09:08.729640    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:09:08.926083 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:09:08.926062    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:09:33.585705 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:09:33.585672    2558 generic.go:356] "Generic (PLEG): container finished" podID="9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f" containerID="b5d60faa7e1f6e69155ce44489faac0fbbc6192cb60c7fb97b3667da6fb8cdfa" exitCode=255
Sep 17 23:09:33.586089 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:09:33.585738    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q" event={"ID":"9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f","Type":"ContainerDied","Data":"b5d60faa7e1f6e69155ce44489faac0fbbc6192cb60c7fb97b3667da6fb8cdfa"}
Sep 17 23:09:33.586089 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:09:33.585995    2558 scope.go:117] "RemoveContainer" containerID="b5d60faa7e1f6e69155ce44489faac0fbbc6192cb60c7fb97b3667da6fb8cdfa"
Sep 17 23:09:34.590307 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:09:34.590272    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c57688c9d-6ww6q" event={"ID":"9b83e0f5-c9a7-47c4-acaa-ee8cadcb525f","Type":"ContainerStarted","Data":"f2696025ce602a6824cd187bcdc3de47d66fa6c1fa375404c03223f1337e1d77"}
Sep 17 23:09:42.140006 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:09:42.139974    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:09.585746 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:09.585667    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:09.686480 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:09.686440    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:12.260228 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:12.260197    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:12.361930 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:12.361903    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:12.481729 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:12.481706    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:12.575643 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:12.575568    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:12.676285 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:12.676260    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:14.267017 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.266987    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:14.317380 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.317344    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:14.619766 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.619699    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-p2x2j"]
Sep 17 23:10:14.621733 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.621716    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:14.624258 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.624232    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 23:10:14.624374 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.624306    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 23:10:14.624505 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.624490    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 23:10:14.624598 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.624545    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qmjfd\""
Sep 17 23:10:14.625426 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.625411    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 23:10:14.631984 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.631965    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p2x2j"]
Sep 17 23:10:14.632997 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.632974    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:14.738390 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.738365    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:14.738488 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.738403    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12d87332-b252-4f33-9f5f-27c8cc74f434-crio-socket\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:14.738488 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.738429    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12d87332-b252-4f33-9f5f-27c8cc74f434-data-volume\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:14.738488 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.738472    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz8cn\" (UniqueName: \"kubernetes.io/projected/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-api-access-wz8cn\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:14.738672 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.738566    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:14.839461 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.839441    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12d87332-b252-4f33-9f5f-27c8cc74f434-crio-socket\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:14.839551 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.839472    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12d87332-b252-4f33-9f5f-27c8cc74f434-data-volume\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:14.839551 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.839489    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz8cn\" (UniqueName: \"kubernetes.io/projected/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-api-access-wz8cn\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:14.839648 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.839568    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12d87332-b252-4f33-9f5f-27c8cc74f434-crio-socket\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:14.839648 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.839599    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:14.839648 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.839643    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:14.839794 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:14.839677    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 23:10:14.839794 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:14.839696    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 23:10:14.839794 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.839726    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12d87332-b252-4f33-9f5f-27c8cc74f434-data-volume\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:14.839794 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:14.839740    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls podName:12d87332-b252-4f33-9f5f-27c8cc74f434 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:15.339724594 +0000 UTC m=+239.520363010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p2x2j" (UID: "12d87332-b252-4f33-9f5f-27c8cc74f434") : secret "insights-runtime-extractor-tls" not found
Sep 17 23:10:14.839794 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:14.839753    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm podName:12d87332-b252-4f33-9f5f-27c8cc74f434 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:15.339747426 +0000 UTC m=+239.520385842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p2x2j" (UID: "12d87332-b252-4f33-9f5f-27c8cc74f434") : configmap "kube-rbac-proxy" not found
Sep 17 23:10:14.847542 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:14.847524    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz8cn\" (UniqueName: \"kubernetes.io/projected/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-api-access-wz8cn\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:15.117290 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:15.117269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:15.342857 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:15.342829    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:15.343162 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:15.342877    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:15.343162 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:15.342971    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 23:10:15.343162 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:15.342976    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 23:10:15.343162 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:15.343031    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm podName:12d87332-b252-4f33-9f5f-27c8cc74f434 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:16.343016075 +0000 UTC m=+240.523654496 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p2x2j" (UID: "12d87332-b252-4f33-9f5f-27c8cc74f434") : configmap "kube-rbac-proxy" not found
Sep 17 23:10:15.343162 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:15.343051    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls podName:12d87332-b252-4f33-9f5f-27c8cc74f434 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:16.343041108 +0000 UTC m=+240.523679541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p2x2j" (UID: "12d87332-b252-4f33-9f5f-27c8cc74f434") : secret "insights-runtime-extractor-tls" not found
Sep 17 23:10:15.351250 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:15.351226    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:15.438143 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:15.438121    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:15.830579 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:15.830478    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:15.887509 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:15.887469    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:16.349702 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:16.349675    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:16.350037 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:16.349723    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:16.350037 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:16.349807    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 23:10:16.350037 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:16.349808    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 23:10:16.350037 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:16.349859    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm podName:12d87332-b252-4f33-9f5f-27c8cc74f434 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:18.34984272 +0000 UTC m=+242.530481135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p2x2j" (UID: "12d87332-b252-4f33-9f5f-27c8cc74f434") : configmap "kube-rbac-proxy" not found
Sep 17 23:10:16.350037 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:16.349873    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls podName:12d87332-b252-4f33-9f5f-27c8cc74f434 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:18.349866936 +0000 UTC m=+242.530505352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p2x2j" (UID: "12d87332-b252-4f33-9f5f-27c8cc74f434") : secret "insights-runtime-extractor-tls" not found
Sep 17 23:10:16.781890 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:16.781865    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nmb7f_0e34c616-c82c-4618-a991-5fb36612f2df/dns-node-resolver/0.log"
Sep 17 23:10:17.381812 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.381790    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cxgg6_6782274a-ac3d-41e1-ad5d-d221b7abb981/node-ca/0.log"
Sep 17 23:10:17.422182 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.422158    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-vlv2d"]
Sep 17 23:10:17.423923 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.423909    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d"
Sep 17 23:10:17.426149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.426135    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 17 23:10:17.427123 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.427106    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 17 23:10:17.427214 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.427105    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-mcgpd\""
Sep 17 23:10:17.427214 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.427110    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 17 23:10:17.427214 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.427111    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 17 23:10:17.432689 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.432661    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-vlv2d"]
Sep 17 23:10:17.456699 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.456677    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:17.456855 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.456839    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c0be8c56-dfab-4b7c-bace-72334b4bc1d6-signing-cabundle\") pod \"service-ca-55f6fc7b86-vlv2d\" (UID: \"c0be8c56-dfab-4b7c-bace-72334b4bc1d6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d"
Sep 17 23:10:17.456907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.456890    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c0be8c56-dfab-4b7c-bace-72334b4bc1d6-signing-key\") pod \"service-ca-55f6fc7b86-vlv2d\" (UID: \"c0be8c56-dfab-4b7c-bace-72334b4bc1d6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d"
Sep 17 23:10:17.456963 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.456950    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jgp7\" (UniqueName: \"kubernetes.io/projected/c0be8c56-dfab-4b7c-bace-72334b4bc1d6-kube-api-access-2jgp7\") pod \"service-ca-55f6fc7b86-vlv2d\" (UID: \"c0be8c56-dfab-4b7c-bace-72334b4bc1d6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d"
Sep 17 23:10:17.516970 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.516928    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:17.557528 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.557495    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jgp7\" (UniqueName: \"kubernetes.io/projected/c0be8c56-dfab-4b7c-bace-72334b4bc1d6-kube-api-access-2jgp7\") pod \"service-ca-55f6fc7b86-vlv2d\" (UID: \"c0be8c56-dfab-4b7c-bace-72334b4bc1d6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d"
Sep 17 23:10:17.557625 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.557554    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c0be8c56-dfab-4b7c-bace-72334b4bc1d6-signing-cabundle\") pod \"service-ca-55f6fc7b86-vlv2d\" (UID: \"c0be8c56-dfab-4b7c-bace-72334b4bc1d6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d"
Sep 17 23:10:17.557691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.557621    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c0be8c56-dfab-4b7c-bace-72334b4bc1d6-signing-key\") pod \"service-ca-55f6fc7b86-vlv2d\" (UID: \"c0be8c56-dfab-4b7c-bace-72334b4bc1d6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d"
Sep 17 23:10:17.558162 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.558143    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c0be8c56-dfab-4b7c-bace-72334b4bc1d6-signing-cabundle\") pod \"service-ca-55f6fc7b86-vlv2d\" (UID: \"c0be8c56-dfab-4b7c-bace-72334b4bc1d6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d"
Sep 17 23:10:17.559838 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.559819    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c0be8c56-dfab-4b7c-bace-72334b4bc1d6-signing-key\") pod \"service-ca-55f6fc7b86-vlv2d\" (UID: \"c0be8c56-dfab-4b7c-bace-72334b4bc1d6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d"
Sep 17 23:10:17.567018 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.567001    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jgp7\" (UniqueName: \"kubernetes.io/projected/c0be8c56-dfab-4b7c-bace-72334b4bc1d6-kube-api-access-2jgp7\") pod \"service-ca-55f6fc7b86-vlv2d\" (UID: \"c0be8c56-dfab-4b7c-bace-72334b4bc1d6\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d"
Sep 17 23:10:17.732378 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.732359    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d"
Sep 17 23:10:17.996209 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:17.996142    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-vlv2d"]
Sep 17 23:10:17.999733 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:10:17.999713    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0be8c56_dfab_4b7c_bace_72334b4bc1d6.slice/crio-fd5edd8ab2ee18f7ba992cae414acb108683fd47b201083a477ebd58d8f90207 WatchSource:0}: Error finding container fd5edd8ab2ee18f7ba992cae414acb108683fd47b201083a477ebd58d8f90207: Status 404 returned error can't find the container with id fd5edd8ab2ee18f7ba992cae414acb108683fd47b201083a477ebd58d8f90207
Sep 17 23:10:18.292093 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:18.292022    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:18.363328 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:18.363303    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:18.363458 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:18.363342    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:18.363545 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:18.363467    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 23:10:18.363545 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:18.363481    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 23:10:18.363648 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:18.363566    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls podName:12d87332-b252-4f33-9f5f-27c8cc74f434 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:22.363544939 +0000 UTC m=+246.544183374 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p2x2j" (UID: "12d87332-b252-4f33-9f5f-27c8cc74f434") : secret "insights-runtime-extractor-tls" not found
Sep 17 23:10:18.363648 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:18.363590    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm podName:12d87332-b252-4f33-9f5f-27c8cc74f434 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:22.363578898 +0000 UTC m=+246.544217314 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p2x2j" (UID: "12d87332-b252-4f33-9f5f-27c8cc74f434") : configmap "kube-rbac-proxy" not found
Sep 17 23:10:18.581605 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:18.581508    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wc6w2_73900d7a-0740-48c9-9a8d-5e576c1316b3/serve-healthcheck-canary/0.log"
Sep 17 23:10:18.709245 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:18.709211    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d" event={"ID":"c0be8c56-dfab-4b7c-bace-72334b4bc1d6","Type":"ContainerStarted","Data":"fd5edd8ab2ee18f7ba992cae414acb108683fd47b201083a477ebd58d8f90207"}
Sep 17 23:10:20.719351 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:20.719291    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d" event={"ID":"c0be8c56-dfab-4b7c-bace-72334b4bc1d6","Type":"ContainerStarted","Data":"3c8d0e543a41faff6429dbb3f42210c02f678273143078829591a7e81a844386"}
Sep 17 23:10:20.735226 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:20.735165    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-vlv2d" podStartSLOduration=1.863466785 podStartE2EDuration="3.735146195s" totalImagesPullingTime="1.87167941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:10:20.73491063 +0000 UTC m=+244.915549069" watchObservedRunningTime="2026-09-17 23:10:20.735146195 +0000 UTC m=+244.915784631"
Sep 17 23:10:20.770794 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:20.770769    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:20.909082 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:20.909052    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:21.054687 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:21.054609    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:21.157085 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:21.157041    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:21.259373 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:21.259327    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:21.361957 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:21.361881    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:21.510993 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:21.510964    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:21.635210 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:21.635139    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:22.075785 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:22.075758    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:22.390239 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:22.390166    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:22.390239 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:22.390226    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:22.390397 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:22.390286    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 23:10:22.390397 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:22.390299    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 23:10:22.390397 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:22.390341    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm podName:12d87332-b252-4f33-9f5f-27c8cc74f434 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:30.390327902 +0000 UTC m=+254.570966318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p2x2j" (UID: "12d87332-b252-4f33-9f5f-27c8cc74f434") : configmap "kube-rbac-proxy" not found
Sep 17 23:10:22.390397 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:22.390354    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls podName:12d87332-b252-4f33-9f5f-27c8cc74f434 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:30.390348143 +0000 UTC m=+254.570986559 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p2x2j" (UID: "12d87332-b252-4f33-9f5f-27c8cc74f434") : secret "insights-runtime-extractor-tls" not found
Sep 17 23:10:22.874659 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:22.874637    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:23.071159 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:23.071134    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:23.475799 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:23.475771    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:23.675701 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:23.675662    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:24.066945 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:24.066911    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:24.675420 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:24.675395    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:25.890691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:25.890655    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:26.267184 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:26.267154    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:26.868705 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:26.868678    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:27.069447 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:27.069421    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:27.277441 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:27.277409    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:27.371115 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:27.371093    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:27.672328 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:27.672260    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:28.467174 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:28.467141    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:28.675660 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:28.675633    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:29.073241 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:29.073206    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:29.279941 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:29.279909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:29.674088 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:29.674059    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:29.840333 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:29.840306    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:10:29.842457 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:29.842438    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/569bb01a-b894-4327-843d-9ed9959567c0-metrics-certs\") pod \"network-metrics-daemon-f4lpx\" (UID: \"569bb01a-b894-4327-843d-9ed9959567c0\") " pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:10:29.928731 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:29.928675    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hrj9x\""
Sep 17 23:10:29.936619 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:29.936605    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f4lpx"
Sep 17 23:10:29.999714 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:29.999686    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:30.201186 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:30.201149    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f4lpx"]
Sep 17 23:10:30.204755 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:10:30.204733    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod569bb01a_b894_4327_843d_9ed9959567c0.slice/crio-bddcf2f1b07afe03cb2d5528145c21c37fc8889489ecfc974c774103c0f6512e WatchSource:0}: Error finding container bddcf2f1b07afe03cb2d5528145c21c37fc8889489ecfc974c774103c0f6512e: Status 404 returned error can't find the container with id bddcf2f1b07afe03cb2d5528145c21c37fc8889489ecfc974c774103c0f6512e
Sep 17 23:10:30.284442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:30.284419    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:30.445850 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:30.445830    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:30.445939 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:30.445867    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:30.445985 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:30.445964    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 23:10:30.446040 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:30.446030    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls podName:12d87332-b252-4f33-9f5f-27c8cc74f434 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:46.446010906 +0000 UTC m=+270.626649324 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p2x2j" (UID: "12d87332-b252-4f33-9f5f-27c8cc74f434") : secret "insights-runtime-extractor-tls" not found
Sep 17 23:10:30.446277 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:30.446256    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12d87332-b252-4f33-9f5f-27c8cc74f434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:30.751634 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:30.751607    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f4lpx" event={"ID":"569bb01a-b894-4327-843d-9ed9959567c0","Type":"ContainerStarted","Data":"bddcf2f1b07afe03cb2d5528145c21c37fc8889489ecfc974c774103c0f6512e"}
Sep 17 23:10:30.875738 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:30.875713    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:31.069701 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:31.069614    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:31.271836 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:31.271812    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:31.471146 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:31.471116    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:31.756346 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:31.756322    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f4lpx" event={"ID":"569bb01a-b894-4327-843d-9ed9959567c0","Type":"ContainerStarted","Data":"73aa940c0c6c078e3fefc55a50148565a7880dea5688136794af26ad23b6eeb3"}
Sep 17 23:10:31.756688 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:31.756405    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f4lpx" event={"ID":"569bb01a-b894-4327-843d-9ed9959567c0","Type":"ContainerStarted","Data":"0e89a138e1e498c7545db88527c490fc69a07e3950d44e0e62070888dc295839"}
Sep 17 23:10:31.772666 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:31.772623    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-f4lpx" podStartSLOduration=253.756762265 podStartE2EDuration="4m14.772608413s" totalImagesPullingTime="1.015846148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:10:31.771369761 +0000 UTC m=+255.952008208" watchObservedRunningTime="2026-09-17 23:10:31.772608413 +0000 UTC m=+255.953246850"
Sep 17 23:10:33.419268 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:33.419233    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:33.495793 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:33.495762    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:33.549947 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:33.549918    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:33.721741 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:33.721715    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:34.099244 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:34.099164    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:34.159278 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:34.159252    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:34.528531 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:34.528486    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:34.629135 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:34.629111    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:34.795110 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:34.795054    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:34.847340 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:34.847318    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:34.914887 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:34.914866    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:34.969886 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:34.969864    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:35.171872 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:35.171814    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:35.305564 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:35.305537    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:35.506807 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:35.506786    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:36.200473 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:36.200446    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:36.252008 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:36.251985    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:36.326235 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:36.326210    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:36.427420 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:36.427392    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:37.994263 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:37.994236    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:38.088283 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:38.088262    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:38.190071 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:38.190045    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:43.778435 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:43.778401    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:43.879187 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:43.879163    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:43.950245 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:43.950220    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:44.561544 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:44.561495    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:45.636071 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:45.636038    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:45.736051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:45.736027    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:46.446797 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:46.446763    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:46.448952 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:46.448931    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12d87332-b252-4f33-9f5f-27c8cc74f434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p2x2j\" (UID: \"12d87332-b252-4f33-9f5f-27c8cc74f434\") " pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:46.733060 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:46.732997    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qmjfd\""
Sep 17 23:10:46.741886 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:46.741870    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p2x2j"
Sep 17 23:10:47.013081 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:47.013011    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p2x2j"]
Sep 17 23:10:47.015830 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:10:47.015805    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12d87332_b252_4f33_9f5f_27c8cc74f434.slice/crio-eac1b539bcb5e23bee2b8ede58c39ddcd9d1e7c2d8e5d8ac3d2b1d9029d61242 WatchSource:0}: Error finding container eac1b539bcb5e23bee2b8ede58c39ddcd9d1e7c2d8e5d8ac3d2b1d9029d61242: Status 404 returned error can't find the container with id eac1b539bcb5e23bee2b8ede58c39ddcd9d1e7c2d8e5d8ac3d2b1d9029d61242
Sep 17 23:10:47.809041 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:47.809013    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p2x2j" event={"ID":"12d87332-b252-4f33-9f5f-27c8cc74f434","Type":"ContainerStarted","Data":"f279485105cc1b1de58485f8a65a4e49edf0fcba0b48bbc59be3277cce7b7227"}
Sep 17 23:10:47.809324 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:47.809072    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p2x2j" event={"ID":"12d87332-b252-4f33-9f5f-27c8cc74f434","Type":"ContainerStarted","Data":"e88e42b2ea7b36bc6405844e5d04d8962bbbfe177435c8e3dfe740782813e808"}
Sep 17 23:10:47.809324 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:47.809105    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p2x2j" event={"ID":"12d87332-b252-4f33-9f5f-27c8cc74f434","Type":"ContainerStarted","Data":"eac1b539bcb5e23bee2b8ede58c39ddcd9d1e7c2d8e5d8ac3d2b1d9029d61242"}
Sep 17 23:10:47.974664 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:47.974562    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:48.035463 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:48.035430    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:48.212086 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:48.212061    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:49.133377 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:49.133344    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:49.817221 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:49.817190    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p2x2j" event={"ID":"12d87332-b252-4f33-9f5f-27c8cc74f434","Type":"ContainerStarted","Data":"f3ab05e62e6a71665625e26a71fd89df2c0933cc8f7fdddae798d7aca9cf7342"}
Sep 17 23:10:49.834392 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:49.834351    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-p2x2j" podStartSLOduration=33.590781547 podStartE2EDuration="35.834337486s" totalImagesPullingTime="2.243555939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:10:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:10:49.832798533 +0000 UTC m=+274.013436970" watchObservedRunningTime="2026-09-17 23:10:49.834337486 +0000 UTC m=+274.014975923"
Sep 17 23:10:50.528027 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:50.527995    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:50.581160 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:50.581131    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:52.208629 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:52.208593    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:54.109804 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.109768    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:54.203549 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.203499    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:54.320417 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.320390    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:54.353367 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.353338    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-spths"]
Sep 17 23:10:54.356484 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.356466    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.361877 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.361817    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 23:10:54.361877 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.361818    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 23:10:54.361877 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.361817    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-n6f65\""
Sep 17 23:10:54.362192 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.362177    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 23:10:54.362278 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.362254    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 23:10:54.362692 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.362676    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 23:10:54.362832 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.362786    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 23:10:54.404097 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.404073    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/709b20a2-d606-4a1b-89ca-6225b474c6aa-root\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.404207 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.404110    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/709b20a2-d606-4a1b-89ca-6225b474c6aa-sys\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.404207 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.404137    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/709b20a2-d606-4a1b-89ca-6225b474c6aa-metrics-client-ca\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.404322 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.404256    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.404322 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.404299    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-tls\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.404430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.404324    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccm56\" (UniqueName: \"kubernetes.io/projected/709b20a2-d606-4a1b-89ca-6225b474c6aa-kube-api-access-ccm56\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.404430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.404362    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.404430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.404395    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-textfile\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.404598 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.404435    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-wtmp\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.467392 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.467364    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:54.505483 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.505462    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-textfile\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.505582 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.505497    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-wtmp\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.505582 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.505541    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/709b20a2-d606-4a1b-89ca-6225b474c6aa-root\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.505582 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.505569    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/709b20a2-d606-4a1b-89ca-6225b474c6aa-sys\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.505682 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.505592    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/709b20a2-d606-4a1b-89ca-6225b474c6aa-metrics-client-ca\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.505682 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.505642    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.505682 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.505678    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-tls\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.505769 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.505701    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccm56\" (UniqueName: \"kubernetes.io/projected/709b20a2-d606-4a1b-89ca-6225b474c6aa-kube-api-access-ccm56\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.505769 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.505734    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.505857 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.505827    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-textfile\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.505925 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.505903    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/709b20a2-d606-4a1b-89ca-6225b474c6aa-root\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.505925 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.505913    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/709b20a2-d606-4a1b-89ca-6225b474c6aa-sys\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.506057 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.506038    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-wtmp\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.506196 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:54.506179    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 17 23:10:54.506283 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.506263    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.506366 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:54.506353    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-tls podName:709b20a2-d606-4a1b-89ca-6225b474c6aa nodeName:}" failed. No retries permitted until 2026-09-17 23:10:55.006331631 +0000 UTC m=+279.186970063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-tls") pod "node-exporter-spths" (UID: "709b20a2-d606-4a1b-89ca-6225b474c6aa") : secret "node-exporter-tls" not found
Sep 17 23:10:54.506432 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.506367    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/709b20a2-d606-4a1b-89ca-6225b474c6aa-metrics-client-ca\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.508234 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.508212    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.513806 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.513780    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccm56\" (UniqueName: \"kubernetes.io/projected/709b20a2-d606-4a1b-89ca-6225b474c6aa-kube-api-access-ccm56\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:54.566870 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.566850    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:54.666471 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.666418    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:54.769554 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.769529    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:54.869189 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.869169    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:54.968203 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:54.968179    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:55.009423 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.009404    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-tls\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:55.011535 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.011504    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/709b20a2-d606-4a1b-89ca-6225b474c6aa-node-exporter-tls\") pod \"node-exporter-spths\" (UID: \"709b20a2-d606-4a1b-89ca-6225b474c6aa\") " pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:55.201154 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.201134    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:55.267531 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.267463    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-spths"
Sep 17 23:10:55.277897 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:10:55.277875    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod709b20a2_d606_4a1b_89ca_6225b474c6aa.slice/crio-64419d2bcf4d479731f85bac88634003f967572294b133e2cb7297c486ffebf5 WatchSource:0}: Error finding container 64419d2bcf4d479731f85bac88634003f967572294b133e2cb7297c486ffebf5: Status 404 returned error can't find the container with id 64419d2bcf4d479731f85bac88634003f967572294b133e2cb7297c486ffebf5
Sep 17 23:10:55.329799 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.329777    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:55.417603 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.417581    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:10:55.419288 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.419269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:55.422005 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.421992    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.424237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.424220    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 23:10:55.424579 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.424565    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-42c2v\""
Sep 17 23:10:55.424579 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.424573    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 23:10:55.424718 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.424671    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 23:10:55.424805 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.424792    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 23:10:55.424886 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.424805    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 23:10:55.424886 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.424844    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 23:10:55.425072 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.425058    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 23:10:55.425072 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.425065    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 23:10:55.425271 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.425257    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 23:10:55.437038 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.437020    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:10:55.512661 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.512639    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1111d626-023b-4ece-866e-d6aab765554c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.512755 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.512668    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-web-config\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.512755 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.512703    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1111d626-023b-4ece-866e-d6aab765554c-config-out\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.512755 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.512737    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.512853 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.512765    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.512853 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.512787    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.512853 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.512806    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.512853 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.512834    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1111d626-023b-4ece-866e-d6aab765554c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.512974 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.512853    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1111d626-023b-4ece-866e-d6aab765554c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.512974 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.512892    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.512974 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.512919    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1111d626-023b-4ece-866e-d6aab765554c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.512974 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.512937    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-config-volume\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.512974 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.512953    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx4fs\" (UniqueName: \"kubernetes.io/projected/1111d626-023b-4ece-866e-d6aab765554c-kube-api-access-fx4fs\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.533312 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.533264    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:55.613372 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.613349    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1111d626-023b-4ece-866e-d6aab765554c-config-out\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.613456 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.613375    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.613456 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.613391    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.613456 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.613407    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.613456 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.613421    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.613456 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.613437    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1111d626-023b-4ece-866e-d6aab765554c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.613732 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.613459    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1111d626-023b-4ece-866e-d6aab765554c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.613732 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.613549    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.613732 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.613589    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1111d626-023b-4ece-866e-d6aab765554c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.613732 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.613618    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-config-volume\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.613732 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.613639    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx4fs\" (UniqueName: \"kubernetes.io/projected/1111d626-023b-4ece-866e-d6aab765554c-kube-api-access-fx4fs\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.613732 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.613664    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1111d626-023b-4ece-866e-d6aab765554c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.613732 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.613692    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-web-config\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.614543 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.614497    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1111d626-023b-4ece-866e-d6aab765554c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.614811 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.614788    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1111d626-023b-4ece-866e-d6aab765554c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.615668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.615642    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1111d626-023b-4ece-866e-d6aab765554c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.616568 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.616543    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.616789 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.616756    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1111d626-023b-4ece-866e-d6aab765554c-config-out\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.617184 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.617159    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.617265 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.617172    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.617265 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.617235    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1111d626-023b-4ece-866e-d6aab765554c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.617265 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.617253    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-config-volume\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.617429 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.617385    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.617779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.617760    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-web-config\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.618113 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.618099    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.623484 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.623465    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx4fs\" (UniqueName: \"kubernetes.io/projected/1111d626-023b-4ece-866e-d6aab765554c-kube-api-access-fx4fs\") pod \"alertmanager-main-0\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.730714 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.730691    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:10:55.836616 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:55.836584    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spths" event={"ID":"709b20a2-d606-4a1b-89ca-6225b474c6aa","Type":"ContainerStarted","Data":"64419d2bcf4d479731f85bac88634003f967572294b133e2cb7297c486ffebf5"}
Sep 17 23:10:56.011200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:56.011173    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:10:56.194355 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:10:56.194327    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1111d626_023b_4ece_866e_d6aab765554c.slice/crio-0b9432f15874f67527f2e4c455f7f254eb8f8d301a4414477aeab5a6a434ee16 WatchSource:0}: Error finding container 0b9432f15874f67527f2e4c455f7f254eb8f8d301a4414477aeab5a6a434ee16: Status 404 returned error can't find the container with id 0b9432f15874f67527f2e4c455f7f254eb8f8d301a4414477aeab5a6a434ee16
Sep 17 23:10:56.840820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:56.840787    2558 generic.go:356] "Generic (PLEG): container finished" podID="709b20a2-d606-4a1b-89ca-6225b474c6aa" containerID="8ccf8333d7de8e8aef9b92ba8338944ff353b7a202f2a92ee285b9abbc662a45" exitCode=0
Sep 17 23:10:56.841225 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:56.840859    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spths" event={"ID":"709b20a2-d606-4a1b-89ca-6225b474c6aa","Type":"ContainerDied","Data":"8ccf8333d7de8e8aef9b92ba8338944ff353b7a202f2a92ee285b9abbc662a45"}
Sep 17 23:10:56.842097 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:56.842077    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerStarted","Data":"0b9432f15874f67527f2e4c455f7f254eb8f8d301a4414477aeab5a6a434ee16"}
Sep 17 23:10:57.069753 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:57.069724    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:57.170756 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:57.170697    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:57.281777 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:57.281750    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:57.335729 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:57.335697    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:57.421473 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:57.421412    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:57.479534 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:57.479481    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-t97lt" podUID="d6c1ec66-3187-4649-9205-78492b636e30"
Sep 17 23:10:57.847387 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:57.847365    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spths" event={"ID":"709b20a2-d606-4a1b-89ca-6225b474c6aa","Type":"ContainerStarted","Data":"902aefbf44076109eb56522ab8ea72f00dd381a4060ec795417213e0be2d7cf6"}
Sep 17 23:10:57.847743 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:57.847409    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spths" event={"ID":"709b20a2-d606-4a1b-89ca-6225b474c6aa","Type":"ContainerStarted","Data":"9b36f4e6737d50d19db39d36884aed33d18dd9b3f2b3d49e6fa022b935bd6d11"}
Sep 17 23:10:57.848956 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:57.848932    2558 generic.go:356] "Generic (PLEG): container finished" podID="1111d626-023b-4ece-866e-d6aab765554c" containerID="18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b" exitCode=0
Sep 17 23:10:57.849044 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:57.849018    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerDied","Data":"18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b"}
Sep 17 23:10:57.849088 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:57.849071    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t97lt"
Sep 17 23:10:57.867398 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:57.867362    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-spths" podStartSLOduration=2.849315889 podStartE2EDuration="3.867350462s" totalImagesPullingTime="957.904343ms" totalInitContainerRuntime="60.13023ms" isStatefulPod=true podCreationTimestamp="2026-09-17 23:10:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:10:57.865588579 +0000 UTC m=+282.046227016" watchObservedRunningTime="2026-09-17 23:10:57.867350462 +0000 UTC m=+282.047988900"
Sep 17 23:10:58.476329 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:58.476299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:59.152774 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.152751    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"]
Sep 17 23:10:59.155011 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.154901    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:59.155718 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.155701    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"
Sep 17 23:10:59.157864 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.157841    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 17 23:10:59.157976 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.157898    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-p8psx\""
Sep 17 23:10:59.164389 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.164366    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"]
Sep 17 23:10:59.241568 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.241540    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pstsf\" (UniqueName: \"kubernetes.io/projected/06b34b93-c3db-4c4e-a2f3-894e3b4093b6-kube-api-access-pstsf\") pod \"monitoring-plugin-5984459ff8-8b4g8\" (UID: \"06b34b93-c3db-4c4e-a2f3-894e3b4093b6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"
Sep 17 23:10:59.241677 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.241622    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06b34b93-c3db-4c4e-a2f3-894e3b4093b6-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-8b4g8\" (UID: \"06b34b93-c3db-4c4e-a2f3-894e3b4093b6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"
Sep 17 23:10:59.245904 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.245885    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:59.342914 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.342892    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06b34b93-c3db-4c4e-a2f3-894e3b4093b6-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-8b4g8\" (UID: \"06b34b93-c3db-4c4e-a2f3-894e3b4093b6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"
Sep 17 23:10:59.343020 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.342975    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pstsf\" (UniqueName: \"kubernetes.io/projected/06b34b93-c3db-4c4e-a2f3-894e3b4093b6-kube-api-access-pstsf\") pod \"monitoring-plugin-5984459ff8-8b4g8\" (UID: \"06b34b93-c3db-4c4e-a2f3-894e3b4093b6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"
Sep 17 23:10:59.343104 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:59.343073    2558 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 17 23:10:59.343174 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:10:59.343145    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06b34b93-c3db-4c4e-a2f3-894e3b4093b6-monitoring-plugin-cert podName:06b34b93-c3db-4c4e-a2f3-894e3b4093b6 nodeName:}" failed. No retries permitted until 2026-09-17 23:10:59.843123825 +0000 UTC m=+284.023762241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/06b34b93-c3db-4c4e-a2f3-894e3b4093b6-monitoring-plugin-cert") pod "monitoring-plugin-5984459ff8-8b4g8" (UID: "06b34b93-c3db-4c4e-a2f3-894e3b4093b6") : secret "monitoring-plugin-cert" not found
Sep 17 23:10:59.353055 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.353028    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pstsf\" (UniqueName: \"kubernetes.io/projected/06b34b93-c3db-4c4e-a2f3-894e3b4093b6-kube-api-access-pstsf\") pod \"monitoring-plugin-5984459ff8-8b4g8\" (UID: \"06b34b93-c3db-4c4e-a2f3-894e3b4093b6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"
Sep 17 23:10:59.515541 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.515498    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:10:59.847802 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.847777    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06b34b93-c3db-4c4e-a2f3-894e3b4093b6-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-8b4g8\" (UID: \"06b34b93-c3db-4c4e-a2f3-894e3b4093b6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"
Sep 17 23:10:59.849980 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.849964    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06b34b93-c3db-4c4e-a2f3-894e3b4093b6-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-8b4g8\" (UID: \"06b34b93-c3db-4c4e-a2f3-894e3b4093b6\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"
Sep 17 23:10:59.858179 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.858155    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerStarted","Data":"35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4"}
Sep 17 23:10:59.858274 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.858215    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerStarted","Data":"4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee"}
Sep 17 23:10:59.858274 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.858240    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerStarted","Data":"9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708"}
Sep 17 23:10:59.858274 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.858265    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerStarted","Data":"ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4"}
Sep 17 23:10:59.858374 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:10:59.858288    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerStarted","Data":"a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6"}
Sep 17 23:11:00.082347 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:00.082316    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"
Sep 17 23:11:00.267669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:00.267646    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:00.351759 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:00.351734    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:00.407095 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:00.407045    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"]
Sep 17 23:11:00.409959 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:11:00.409940    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06b34b93_c3db_4c4e_a2f3_894e3b4093b6.slice/crio-be863d315cacb6b574e61978d290a7b2f066a972433dddeff5ba6f57690ec4aa WatchSource:0}: Error finding container be863d315cacb6b574e61978d290a7b2f066a972433dddeff5ba6f57690ec4aa: Status 404 returned error can't find the container with id be863d315cacb6b574e61978d290a7b2f066a972433dddeff5ba6f57690ec4aa
Sep 17 23:11:00.414641 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:00.414616    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:00.490767 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:00.490739    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:00.560170 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:00.560137    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:00.864246 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:00.864216    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerStarted","Data":"390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979"}
Sep 17 23:11:00.865357 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:00.865335    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8" event={"ID":"06b34b93-c3db-4c4e-a2f3-894e3b4093b6","Type":"ContainerStarted","Data":"be863d315cacb6b574e61978d290a7b2f066a972433dddeff5ba6f57690ec4aa"}
Sep 17 23:11:00.889183 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:00.889148    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.332683644 podStartE2EDuration="5.889133968s" totalImagesPullingTime="3.540362816s" totalInitContainerRuntime="16.087508ms" isStatefulPod=false podCreationTimestamp="2026-09-17 23:10:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:00.887776336 +0000 UTC m=+285.068414773" watchObservedRunningTime="2026-09-17 23:11:00.889133968 +0000 UTC m=+285.069772406"
Sep 17 23:11:01.157634 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:01.157557    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:11:01.160259 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:01.160231    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6c1ec66-3187-4649-9205-78492b636e30-metrics-tls\") pod \"dns-default-t97lt\" (UID: \"d6c1ec66-3187-4649-9205-78492b636e30\") " pod="openshift-dns/dns-default-t97lt"
Sep 17 23:11:01.451697 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:01.451668    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7hp2b\""
Sep 17 23:11:01.460458 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:01.460436    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t97lt"
Sep 17 23:11:01.869944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:01.869904    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8" event={"ID":"06b34b93-c3db-4c4e-a2f3-894e3b4093b6","Type":"ContainerStarted","Data":"872e014600361b8618298002e4d7603fc6871ca85dd4162eeec70a845efa9c13"}
Sep 17 23:11:01.870294 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:01.870277    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"
Sep 17 23:11:01.874721 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:01.874700    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8"
Sep 17 23:11:01.884656 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:01.884621    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-8b4g8" podStartSLOduration=1.53913551 podStartE2EDuration="2.884608198s" totalImagesPullingTime="1.345472688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:01.883568092 +0000 UTC m=+286.064206529" watchObservedRunningTime="2026-09-17 23:11:01.884608198 +0000 UTC m=+286.065246635"
Sep 17 23:11:01.965931 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:01.965850    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t97lt"]
Sep 17 23:11:01.969528 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:11:01.969492    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c1ec66_3187_4649_9205_78492b636e30.slice/crio-8c970a1afd638ff4c51f470a5536c8d52776b5d9c3952cf359157ab8f610737e WatchSource:0}: Error finding container 8c970a1afd638ff4c51f470a5536c8d52776b5d9c3952cf359157ab8f610737e: Status 404 returned error can't find the container with id 8c970a1afd638ff4c51f470a5536c8d52776b5d9c3952cf359157ab8f610737e
Sep 17 23:11:02.875374 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:02.875335    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t97lt" event={"ID":"d6c1ec66-3187-4649-9205-78492b636e30","Type":"ContainerStarted","Data":"8c970a1afd638ff4c51f470a5536c8d52776b5d9c3952cf359157ab8f610737e"}
Sep 17 23:11:03.810711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.810632    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:03.842441 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.842412    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b55c867cd-p86bz"]
Sep 17 23:11:03.845455 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.845440    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:03.847803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.847775    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 23:11:03.847916 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.847837    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 23:11:03.847916 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.847887    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 23:11:03.847916 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.847896    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 23:11:03.848084 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.847934    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 23:11:03.848157 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.848144    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 23:11:03.848231 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.848216    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 23:11:03.848287 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.848240    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rk242\""
Sep 17 23:11:03.854024 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.854003    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 23:11:03.857285 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.857264    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b55c867cd-p86bz"]
Sep 17 23:11:03.879965 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.879934    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:03.880570 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.880553    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t97lt" event={"ID":"d6c1ec66-3187-4649-9205-78492b636e30","Type":"ContainerStarted","Data":"cc567ddf1f7f245df56c1b3befc62b16acda84a37a83f52dbfbdc103012efe41"}
Sep 17 23:11:03.880652 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.880602    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t97lt" event={"ID":"d6c1ec66-3187-4649-9205-78492b636e30","Type":"ContainerStarted","Data":"12eb955b5c78a9a334fd267e67f40c5cb651430039095a8fa3ba5e5376edef54"}
Sep 17 23:11:03.897443 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.897398    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-t97lt" podStartSLOduration=251.386487507 podStartE2EDuration="4m12.897382825s" totalImagesPullingTime="1.510895318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:03.896930189 +0000 UTC m=+288.077568627" watchObservedRunningTime="2026-09-17 23:11:03.897382825 +0000 UTC m=+288.078021267"
Sep 17 23:11:03.982670 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.982647    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-console-config\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:03.982740 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.982690    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-service-ca\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:03.982829 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.982804    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-trusted-ca-bundle\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:03.982954 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.982861    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a99f446-1c86-438f-abd9-4063d8f813bc-console-oauth-config\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:03.982954 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.982884    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47p8m\" (UniqueName: \"kubernetes.io/projected/5a99f446-1c86-438f-abd9-4063d8f813bc-kube-api-access-47p8m\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:03.982954 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.982911    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a99f446-1c86-438f-abd9-4063d8f813bc-console-serving-cert\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:03.983099 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:03.982992    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-oauth-serving-cert\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.083472 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.083427    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-trusted-ca-bundle\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.083472 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.083453    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a99f446-1c86-438f-abd9-4063d8f813bc-console-oauth-config\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.083472 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.083469    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47p8m\" (UniqueName: \"kubernetes.io/projected/5a99f446-1c86-438f-abd9-4063d8f813bc-kube-api-access-47p8m\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.083684 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.083498    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a99f446-1c86-438f-abd9-4063d8f813bc-console-serving-cert\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.083684 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.083648    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-oauth-serving-cert\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.083790 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.083730    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-console-config\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.083790 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.083774    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-service-ca\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.084427 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.084406    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-trusted-ca-bundle\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.084535 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.084494    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-oauth-serving-cert\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.084590 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.084559    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-console-config\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.084852 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.084831    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-service-ca\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.086009 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.085980    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a99f446-1c86-438f-abd9-4063d8f813bc-console-oauth-config\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.086170 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.086155    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a99f446-1c86-438f-abd9-4063d8f813bc-console-serving-cert\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.090800 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.090784    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47p8m\" (UniqueName: \"kubernetes.io/projected/5a99f446-1c86-438f-abd9-4063d8f813bc-kube-api-access-47p8m\") pod \"console-5b55c867cd-p86bz\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") " pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.154994 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.154969    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:04.422323 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.422300    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b55c867cd-p86bz"]
Sep 17 23:11:04.424708 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:11:04.424677    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a99f446_1c86_438f_abd9_4063d8f813bc.slice/crio-1e21d341556de259b18bd556e5aeaa84a6b563217ef31f7225e349209b69a1b7 WatchSource:0}: Error finding container 1e21d341556de259b18bd556e5aeaa84a6b563217ef31f7225e349209b69a1b7: Status 404 returned error can't find the container with id 1e21d341556de259b18bd556e5aeaa84a6b563217ef31f7225e349209b69a1b7
Sep 17 23:11:04.766257 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.766219    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:04.865375 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.865300    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:04.891293 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.891198    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b55c867cd-p86bz" event={"ID":"5a99f446-1c86-438f-abd9-4063d8f813bc","Type":"ContainerStarted","Data":"1e21d341556de259b18bd556e5aeaa84a6b563217ef31f7225e349209b69a1b7"}
Sep 17 23:11:04.891293 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:04.891247    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-t97lt"
Sep 17 23:11:05.726724 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:05.726671    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:08.744160 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:08.744109    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw" podUID="992521db-fb70-4902-bc57-5d05455e1fe8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 17 23:11:08.906737 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:08.906707    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b55c867cd-p86bz" event={"ID":"5a99f446-1c86-438f-abd9-4063d8f813bc","Type":"ContainerStarted","Data":"006fe630612396ca38e94ab1e6b531c1c168c56c45494ca592a50ee7deab5cc4"}
Sep 17 23:11:08.925786 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:08.925741    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b55c867cd-p86bz" podStartSLOduration=2.213632791 podStartE2EDuration="5.925724227s" totalImagesPullingTime="3.712091436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:11:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:08.925280475 +0000 UTC m=+293.105918916" watchObservedRunningTime="2026-09-17 23:11:08.925724227 +0000 UTC m=+293.106362669"
Sep 17 23:11:09.514741 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:09.514713    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:10.522170 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:10.522140    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:10.570961 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:10.570933    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:13.117449 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:13.117419    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:13.219107 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:13.219067    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:14.156077 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:14.156040    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:14.156435 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:14.156124    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:14.160718 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:14.160698    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:14.663969 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:14.663941    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:14.766292 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:14.766262    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:14.897096 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:14.897067    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t97lt"
Sep 17 23:11:14.931030 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:14.930975    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:11:15.079830 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:15.079805    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:15.180332 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:15.180303    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:15.280094 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:15.280068    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:17.804615 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:17.804586    2558 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 23:11:18.743829 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:18.743786    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw" podUID="992521db-fb70-4902-bc57-5d05455e1fe8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 17 23:11:19.595423 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:19.595390    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:19.699457 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:19.699430    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:19.796689 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:19.796660    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:19.896668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:19.896616    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:24.017070 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:24.017032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:28.743910 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:28.743870    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw" podUID="992521db-fb70-4902-bc57-5d05455e1fe8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 17 23:11:28.744279 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:28.743946    2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw"
Sep 17 23:11:28.744421 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:28.744391    2558 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"48401d9515a476161626f0e3a488e5160d53164c9c2ff003c9e2a09d8d332817"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 17 23:11:28.744457 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:28.744442    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw" podUID="992521db-fb70-4902-bc57-5d05455e1fe8" containerName="service-proxy" containerID="cri-o://48401d9515a476161626f0e3a488e5160d53164c9c2ff003c9e2a09d8d332817" gracePeriod=30
Sep 17 23:11:28.974189 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:28.974162    2558 generic.go:356] "Generic (PLEG): container finished" podID="992521db-fb70-4902-bc57-5d05455e1fe8" containerID="48401d9515a476161626f0e3a488e5160d53164c9c2ff003c9e2a09d8d332817" exitCode=2
Sep 17 23:11:28.974287 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:28.974197    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw" event={"ID":"992521db-fb70-4902-bc57-5d05455e1fe8","Type":"ContainerDied","Data":"48401d9515a476161626f0e3a488e5160d53164c9c2ff003c9e2a09d8d332817"}
Sep 17 23:11:28.974287 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:28.974223    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb68dbc8-67nnw" event={"ID":"992521db-fb70-4902-bc57-5d05455e1fe8","Type":"ContainerStarted","Data":"f8845db91d5e510f13b8b93f36587936273f6421f0fe879e16eab28b9196c6f7"}
Sep 17 23:11:34.797019 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:34.796984    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:11:34.855866 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:11:34.855837    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:10.441882 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:10.441837    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:10.497272 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:10.497244    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:10.980887 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:10.980859    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:11.033432 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:11.033401    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:14.666335 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:14.666301    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:12:14.666747 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:14.666625    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="alertmanager" containerID="cri-o://a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6" gracePeriod=120
Sep 17 23:12:14.666747 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:14.666687    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="kube-rbac-proxy-web" containerID="cri-o://9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708" gracePeriod=120
Sep 17 23:12:14.666747 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:14.666710    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="prom-label-proxy" containerID="cri-o://390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979" gracePeriod=120
Sep 17 23:12:14.666920 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:14.666730    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="config-reloader" containerID="cri-o://ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4" gracePeriod=120
Sep 17 23:12:14.666920 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:14.666760    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="kube-rbac-proxy" containerID="cri-o://4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee" gracePeriod=120
Sep 17 23:12:14.666920 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:14.666685    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="kube-rbac-proxy-metric" containerID="cri-o://35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4" gracePeriod=120
Sep 17 23:12:15.076232 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.076186    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:15.131955 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.131921    2558 generic.go:356] "Generic (PLEG): container finished" podID="1111d626-023b-4ece-866e-d6aab765554c" containerID="390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979" exitCode=0
Sep 17 23:12:15.131955 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.131945    2558 generic.go:356] "Generic (PLEG): container finished" podID="1111d626-023b-4ece-866e-d6aab765554c" containerID="4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee" exitCode=0
Sep 17 23:12:15.131955 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.131951    2558 generic.go:356] "Generic (PLEG): container finished" podID="1111d626-023b-4ece-866e-d6aab765554c" containerID="ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4" exitCode=0
Sep 17 23:12:15.131955 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.131957    2558 generic.go:356] "Generic (PLEG): container finished" podID="1111d626-023b-4ece-866e-d6aab765554c" containerID="a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6" exitCode=0
Sep 17 23:12:15.132267 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.132001    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerDied","Data":"390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979"}
Sep 17 23:12:15.132267 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.132114    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerDied","Data":"4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee"}
Sep 17 23:12:15.132267 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.132158    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerDied","Data":"ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4"}
Sep 17 23:12:15.132267 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.132199    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerDied","Data":"a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6"}
Sep 17 23:12:15.135099 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.135079    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:15.900092 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.900065    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:15.979030 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.978946    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1111d626-023b-4ece-866e-d6aab765554c\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") "
Sep 17 23:12:15.979030 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.978997    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy\") pod \"1111d626-023b-4ece-866e-d6aab765554c\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") "
Sep 17 23:12:15.979349 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979036    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-main-tls\") pod \"1111d626-023b-4ece-866e-d6aab765554c\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") "
Sep 17 23:12:15.979349 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979061    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1111d626-023b-4ece-866e-d6aab765554c-tls-assets\") pod \"1111d626-023b-4ece-866e-d6aab765554c\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") "
Sep 17 23:12:15.979349 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979084    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1111d626-023b-4ece-866e-d6aab765554c-alertmanager-trusted-ca-bundle\") pod \"1111d626-023b-4ece-866e-d6aab765554c\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") "
Sep 17 23:12:15.979349 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979115    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1111d626-023b-4ece-866e-d6aab765554c-alertmanager-main-db\") pod \"1111d626-023b-4ece-866e-d6aab765554c\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") "
Sep 17 23:12:15.979590 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979388    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-web-config\") pod \"1111d626-023b-4ece-866e-d6aab765554c\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") "
Sep 17 23:12:15.979590 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979440    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fx4fs\" (UniqueName: \"kubernetes.io/projected/1111d626-023b-4ece-866e-d6aab765554c-kube-api-access-fx4fs\") pod \"1111d626-023b-4ece-866e-d6aab765554c\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") "
Sep 17 23:12:15.979590 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979491    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-config-volume\") pod \"1111d626-023b-4ece-866e-d6aab765554c\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") "
Sep 17 23:12:15.979590 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979546    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1111d626-023b-4ece-866e-d6aab765554c-config-out\") pod \"1111d626-023b-4ece-866e-d6aab765554c\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") "
Sep 17 23:12:15.979590 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979438    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1111d626-023b-4ece-866e-d6aab765554c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1111d626-023b-4ece-866e-d6aab765554c" (UID: "1111d626-023b-4ece-866e-d6aab765554c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:12:15.979590 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979577    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy-web\") pod \"1111d626-023b-4ece-866e-d6aab765554c\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") "
Sep 17 23:12:15.979590 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979537    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1111d626-023b-4ece-866e-d6aab765554c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1111d626-023b-4ece-866e-d6aab765554c" (UID: "1111d626-023b-4ece-866e-d6aab765554c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:12:15.979952 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979605    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-cluster-tls-config\") pod \"1111d626-023b-4ece-866e-d6aab765554c\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") "
Sep 17 23:12:15.979952 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979637    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1111d626-023b-4ece-866e-d6aab765554c-metrics-client-ca\") pod \"1111d626-023b-4ece-866e-d6aab765554c\" (UID: \"1111d626-023b-4ece-866e-d6aab765554c\") "
Sep 17 23:12:15.979952 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979867    2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1111d626-023b-4ece-866e-d6aab765554c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:15.979952 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.979884    2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1111d626-023b-4ece-866e-d6aab765554c-alertmanager-main-db\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:15.980243 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.980170    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1111d626-023b-4ece-866e-d6aab765554c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1111d626-023b-4ece-866e-d6aab765554c" (UID: "1111d626-023b-4ece-866e-d6aab765554c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:12:15.983093 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.983048    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1111d626-023b-4ece-866e-d6aab765554c" (UID: "1111d626-023b-4ece-866e-d6aab765554c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:12:15.983445 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.983263    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1111d626-023b-4ece-866e-d6aab765554c-config-out" (OuterVolumeSpecName: "config-out") pod "1111d626-023b-4ece-866e-d6aab765554c" (UID: "1111d626-023b-4ece-866e-d6aab765554c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:12:15.983445 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.983339    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1111d626-023b-4ece-866e-d6aab765554c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1111d626-023b-4ece-866e-d6aab765554c" (UID: "1111d626-023b-4ece-866e-d6aab765554c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:12:15.983445 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.983333    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1111d626-023b-4ece-866e-d6aab765554c-kube-api-access-fx4fs" (OuterVolumeSpecName: "kube-api-access-fx4fs") pod "1111d626-023b-4ece-866e-d6aab765554c" (UID: "1111d626-023b-4ece-866e-d6aab765554c"). InnerVolumeSpecName "kube-api-access-fx4fs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:12:15.983445 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.983411    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-config-volume" (OuterVolumeSpecName: "config-volume") pod "1111d626-023b-4ece-866e-d6aab765554c" (UID: "1111d626-023b-4ece-866e-d6aab765554c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:12:15.983683 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.983459    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1111d626-023b-4ece-866e-d6aab765554c" (UID: "1111d626-023b-4ece-866e-d6aab765554c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:12:15.983792 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.983770    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1111d626-023b-4ece-866e-d6aab765554c" (UID: "1111d626-023b-4ece-866e-d6aab765554c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:12:15.984124 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.984098    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1111d626-023b-4ece-866e-d6aab765554c" (UID: "1111d626-023b-4ece-866e-d6aab765554c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:12:15.987542 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.987497    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1111d626-023b-4ece-866e-d6aab765554c" (UID: "1111d626-023b-4ece-866e-d6aab765554c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:12:15.994270 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:15.994242    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-web-config" (OuterVolumeSpecName: "web-config") pod "1111d626-023b-4ece-866e-d6aab765554c" (UID: "1111d626-023b-4ece-866e-d6aab765554c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:12:16.080964 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.080914    2558 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-web-config\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:16.080964 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.080956    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fx4fs\" (UniqueName: \"kubernetes.io/projected/1111d626-023b-4ece-866e-d6aab765554c-kube-api-access-fx4fs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:16.080964 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.080967    2558 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-config-volume\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:16.080964 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.080975    2558 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1111d626-023b-4ece-866e-d6aab765554c-config-out\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:16.080964 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.080984    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:16.081254 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.080992    2558 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-cluster-tls-config\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:16.081254 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.081000    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1111d626-023b-4ece-866e-d6aab765554c-metrics-client-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:16.081254 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.081009    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:16.081254 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.081019    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:16.081254 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.081027    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1111d626-023b-4ece-866e-d6aab765554c-secret-alertmanager-main-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:16.081254 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.081035    2558 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1111d626-023b-4ece-866e-d6aab765554c-tls-assets\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:16.138363 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.138331    2558 generic.go:356] "Generic (PLEG): container finished" podID="1111d626-023b-4ece-866e-d6aab765554c" containerID="35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4" exitCode=0
Sep 17 23:12:16.138363 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.138356    2558 generic.go:356] "Generic (PLEG): container finished" podID="1111d626-023b-4ece-866e-d6aab765554c" containerID="9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708" exitCode=0
Sep 17 23:12:16.138586 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.138407    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.138586 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.138408    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerDied","Data":"35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4"}
Sep 17 23:12:16.138586 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.138478    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerDied","Data":"9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708"}
Sep 17 23:12:16.138586 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.138504    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1111d626-023b-4ece-866e-d6aab765554c","Type":"ContainerDied","Data":"0b9432f15874f67527f2e4c455f7f254eb8f8d301a4414477aeab5a6a434ee16"}
Sep 17 23:12:16.138586 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.138542    2558 scope.go:117] "RemoveContainer" containerID="390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979"
Sep 17 23:12:16.146009 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.145992    2558 scope.go:117] "RemoveContainer" containerID="35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4"
Sep 17 23:12:16.153023 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.153007    2558 scope.go:117] "RemoveContainer" containerID="4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee"
Sep 17 23:12:16.159466 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.159449    2558 scope.go:117] "RemoveContainer" containerID="9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708"
Sep 17 23:12:16.161653 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.161633    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:12:16.166717 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.166696    2558 scope.go:117] "RemoveContainer" containerID="ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4"
Sep 17 23:12:16.168415 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.168394    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:12:16.173587 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.173570    2558 scope.go:117] "RemoveContainer" containerID="a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6"
Sep 17 23:12:16.180894 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.180877    2558 scope.go:117] "RemoveContainer" containerID="18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b"
Sep 17 23:12:16.187445 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.187428    2558 scope.go:117] "RemoveContainer" containerID="390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979"
Sep 17 23:12:16.187749 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:12:16.187725    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979\": container with ID starting with 390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979 not found: ID does not exist" containerID="390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979"
Sep 17 23:12:16.187840 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.187755    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979"} err="failed to get container status \"390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979\": rpc error: code = NotFound desc = could not find container \"390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979\": container with ID starting with 390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979 not found: ID does not exist"
Sep 17 23:12:16.187840 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.187776    2558 scope.go:117] "RemoveContainer" containerID="35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4"
Sep 17 23:12:16.188016 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:12:16.187998    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4\": container with ID starting with 35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4 not found: ID does not exist" containerID="35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4"
Sep 17 23:12:16.188059 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.188021    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4"} err="failed to get container status \"35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4\": rpc error: code = NotFound desc = could not find container \"35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4\": container with ID starting with 35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4 not found: ID does not exist"
Sep 17 23:12:16.188059 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.188039    2558 scope.go:117] "RemoveContainer" containerID="4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee"
Sep 17 23:12:16.188247 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:12:16.188233    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee\": container with ID starting with 4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee not found: ID does not exist" containerID="4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee"
Sep 17 23:12:16.188286 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.188250    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee"} err="failed to get container status \"4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee\": rpc error: code = NotFound desc = could not find container \"4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee\": container with ID starting with 4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee not found: ID does not exist"
Sep 17 23:12:16.188286 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.188262    2558 scope.go:117] "RemoveContainer" containerID="9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708"
Sep 17 23:12:16.188460 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:12:16.188444    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708\": container with ID starting with 9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708 not found: ID does not exist" containerID="9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708"
Sep 17 23:12:16.188504 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.188464    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708"} err="failed to get container status \"9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708\": rpc error: code = NotFound desc = could not find container \"9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708\": container with ID starting with 9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708 not found: ID does not exist"
Sep 17 23:12:16.188504 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.188476    2558 scope.go:117] "RemoveContainer" containerID="ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4"
Sep 17 23:12:16.188698 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:12:16.188680    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4\": container with ID starting with ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4 not found: ID does not exist" containerID="ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4"
Sep 17 23:12:16.188741 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.188702    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4"} err="failed to get container status \"ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4\": rpc error: code = NotFound desc = could not find container \"ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4\": container with ID starting with ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4 not found: ID does not exist"
Sep 17 23:12:16.188741 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.188714    2558 scope.go:117] "RemoveContainer" containerID="a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6"
Sep 17 23:12:16.188923 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:12:16.188908    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6\": container with ID starting with a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6 not found: ID does not exist" containerID="a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6"
Sep 17 23:12:16.188961 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.188926    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6"} err="failed to get container status \"a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6\": rpc error: code = NotFound desc = could not find container \"a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6\": container with ID starting with a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6 not found: ID does not exist"
Sep 17 23:12:16.188961 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.188938    2558 scope.go:117] "RemoveContainer" containerID="18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b"
Sep 17 23:12:16.189142 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:12:16.189128    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b\": container with ID starting with 18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b not found: ID does not exist" containerID="18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b"
Sep 17 23:12:16.189183 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.189145    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b"} err="failed to get container status \"18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b\": rpc error: code = NotFound desc = could not find container \"18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b\": container with ID starting with 18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b not found: ID does not exist"
Sep 17 23:12:16.189183 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.189156    2558 scope.go:117] "RemoveContainer" containerID="390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979"
Sep 17 23:12:16.189334 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.189320    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979"} err="failed to get container status \"390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979\": rpc error: code = NotFound desc = could not find container \"390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979\": container with ID starting with 390b22007232170ab658ea015738ed5eeec21daeb2ca51e6274caf215d711979 not found: ID does not exist"
Sep 17 23:12:16.189334 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.189334    2558 scope.go:117] "RemoveContainer" containerID="35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4"
Sep 17 23:12:16.189561 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.189499    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4"} err="failed to get container status \"35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4\": rpc error: code = NotFound desc = could not find container \"35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4\": container with ID starting with 35292039ac9c57ddc6e944912cd9695569271a0a3410a5016239da74f6394da4 not found: ID does not exist"
Sep 17 23:12:16.189650 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.189581    2558 scope.go:117] "RemoveContainer" containerID="4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee"
Sep 17 23:12:16.189771 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.189755    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee"} err="failed to get container status \"4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee\": rpc error: code = NotFound desc = could not find container \"4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee\": container with ID starting with 4fa0d30b84dda257e94dff269bca345e8116f79b4727d863bff5bbd4962e06ee not found: ID does not exist"
Sep 17 23:12:16.189815 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.189771    2558 scope.go:117] "RemoveContainer" containerID="9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708"
Sep 17 23:12:16.190001 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.189983    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708"} err="failed to get container status \"9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708\": rpc error: code = NotFound desc = could not find container \"9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708\": container with ID starting with 9510fd79bc6c49b9bb0e7fb08da03885fad3d0356490eed292970387c2f78708 not found: ID does not exist"
Sep 17 23:12:16.190076 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.190002    2558 scope.go:117] "RemoveContainer" containerID="ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4"
Sep 17 23:12:16.190205 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.190185    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4"} err="failed to get container status \"ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4\": rpc error: code = NotFound desc = could not find container \"ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4\": container with ID starting with ad274db8d401790ba41ce7ae320ec5700906227f64ffe513f36495d0cf0836a4 not found: ID does not exist"
Sep 17 23:12:16.190249 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.190206    2558 scope.go:117] "RemoveContainer" containerID="a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6"
Sep 17 23:12:16.190393 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.190376    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6"} err="failed to get container status \"a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6\": rpc error: code = NotFound desc = could not find container \"a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6\": container with ID starting with a124d22bcb519a50f8b3d5c356b05d80585745b8eed378cb709a96c19826e5b6 not found: ID does not exist"
Sep 17 23:12:16.190464 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.190394    2558 scope.go:117] "RemoveContainer" containerID="18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b"
Sep 17 23:12:16.190606 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.190586    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b"} err="failed to get container status \"18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b\": rpc error: code = NotFound desc = could not find container \"18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b\": container with ID starting with 18da7e1938e0a94cd486be3c5803984fb43e07f99dd32e793b0a603ccfc6127b not found: ID does not exist"
Sep 17 23:12:16.196173 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196151    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:12:16.196396 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196383    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="kube-rbac-proxy-web"
Sep 17 23:12:16.196442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196398    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="kube-rbac-proxy-web"
Sep 17 23:12:16.196442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196409    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="kube-rbac-proxy"
Sep 17 23:12:16.196442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196414    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="kube-rbac-proxy"
Sep 17 23:12:16.196442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196422    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="kube-rbac-proxy-metric"
Sep 17 23:12:16.196442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196427    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="kube-rbac-proxy-metric"
Sep 17 23:12:16.196442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196434    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="prom-label-proxy"
Sep 17 23:12:16.196442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196443    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="prom-label-proxy"
Sep 17 23:12:16.196668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196453    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="init-config-reloader"
Sep 17 23:12:16.196668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196459    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="init-config-reloader"
Sep 17 23:12:16.196668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196465    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="config-reloader"
Sep 17 23:12:16.196668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196471    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="config-reloader"
Sep 17 23:12:16.196668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196476    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="alertmanager"
Sep 17 23:12:16.196668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196481    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="alertmanager"
Sep 17 23:12:16.196668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196548    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="kube-rbac-proxy-metric"
Sep 17 23:12:16.196668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196555    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="alertmanager"
Sep 17 23:12:16.196668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196561    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="kube-rbac-proxy-web"
Sep 17 23:12:16.196668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196567    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="config-reloader"
Sep 17 23:12:16.196668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196572    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="prom-label-proxy"
Sep 17 23:12:16.196668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.196579    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1111d626-023b-4ece-866e-d6aab765554c" containerName="kube-rbac-proxy"
Sep 17 23:12:16.199675 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.199661    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.201969 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.201947    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 23:12:16.201969 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.201966    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 23:12:16.202111 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.201977    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 23:12:16.202111 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.201999    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 23:12:16.202214 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.202197    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 23:12:16.202268 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.202220    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 23:12:16.202553 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.202495    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 23:12:16.202553 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.202530    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 23:12:16.202794 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.202780    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-42c2v\""
Sep 17 23:12:16.208139 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.208120    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 23:12:16.215569 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.215550    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:12:16.282433 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.282346    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.282433 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.282392    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-web-config\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.282433 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.282411    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.282711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.282494    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5c479e3-8a7c-4388-b45b-0483ff8781e6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.282711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.282556    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5c479e3-8a7c-4388-b45b-0483ff8781e6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.282711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.282593    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5c479e3-8a7c-4388-b45b-0483ff8781e6-config-out\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.282711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.282615    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-config-volume\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.282711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.282635    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.282711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.282660    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24hmd\" (UniqueName: \"kubernetes.io/projected/d5c479e3-8a7c-4388-b45b-0483ff8781e6-kube-api-access-24hmd\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.282711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.282686    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.282711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.282708    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.282969 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.282723    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d5c479e3-8a7c-4388-b45b-0483ff8781e6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.282969 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.282774    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c479e3-8a7c-4388-b45b-0483ff8781e6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.383719 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.383685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5c479e3-8a7c-4388-b45b-0483ff8781e6-config-out\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.383719 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.383722    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-config-volume\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.383962 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.383742    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.383962 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.383762    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24hmd\" (UniqueName: \"kubernetes.io/projected/d5c479e3-8a7c-4388-b45b-0483ff8781e6-kube-api-access-24hmd\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.383962 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.383791    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.383962 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.383809    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.383962 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.383829    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d5c479e3-8a7c-4388-b45b-0483ff8781e6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.383962 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.383867    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c479e3-8a7c-4388-b45b-0483ff8781e6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.383962 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.383895    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.383962 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.383925    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-web-config\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.383962 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.383951    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.384388 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.383975    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5c479e3-8a7c-4388-b45b-0483ff8781e6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.384388 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.383999    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5c479e3-8a7c-4388-b45b-0483ff8781e6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.384740 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.384713    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d5c479e3-8a7c-4388-b45b-0483ff8781e6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.384831 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.384772    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5c479e3-8a7c-4388-b45b-0483ff8781e6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.385744 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.385713    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c479e3-8a7c-4388-b45b-0483ff8781e6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.386997 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.386737    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.386997 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.386791    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5c479e3-8a7c-4388-b45b-0483ff8781e6-config-out\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.386997 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.386943    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.386997 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.386950    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-config-volume\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.386997 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.386968    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.387387 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.387368    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.387430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.387416    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-web-config\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.387865 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.387848    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d5c479e3-8a7c-4388-b45b-0483ff8781e6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.388730 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.388709    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5c479e3-8a7c-4388-b45b-0483ff8781e6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.393041 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.393024    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24hmd\" (UniqueName: \"kubernetes.io/projected/d5c479e3-8a7c-4388-b45b-0483ff8781e6-kube-api-access-24hmd\") pod \"alertmanager-main-0\" (UID: \"d5c479e3-8a7c-4388-b45b-0483ff8781e6\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.508880 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.508840    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:12:16.783029 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:16.782901    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:12:16.785285 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:12:16.785257    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5c479e3_8a7c_4388_b45b_0483ff8781e6.slice/crio-9e312eb4d80d9676f8e9794e79eba164f94f2f819bfeec59818bae85fb61ce0e WatchSource:0}: Error finding container 9e312eb4d80d9676f8e9794e79eba164f94f2f819bfeec59818bae85fb61ce0e: Status 404 returned error can't find the container with id 9e312eb4d80d9676f8e9794e79eba164f94f2f819bfeec59818bae85fb61ce0e
Sep 17 23:12:17.143986 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:17.143901    2558 generic.go:356] "Generic (PLEG): container finished" podID="d5c479e3-8a7c-4388-b45b-0483ff8781e6" containerID="4213ecaed4114dc2c55439e3aa8641136de91d2fa9e63a043e203595f8c3635d" exitCode=0
Sep 17 23:12:17.143986 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:17.143962    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c479e3-8a7c-4388-b45b-0483ff8781e6","Type":"ContainerDied","Data":"4213ecaed4114dc2c55439e3aa8641136de91d2fa9e63a043e203595f8c3635d"}
Sep 17 23:12:17.144350 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:17.144015    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c479e3-8a7c-4388-b45b-0483ff8781e6","Type":"ContainerStarted","Data":"9e312eb4d80d9676f8e9794e79eba164f94f2f819bfeec59818bae85fb61ce0e"}
Sep 17 23:12:17.808453 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:17.808427    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:17.869354 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:17.869322    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:17.930823 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:17.930796    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1111d626-023b-4ece-866e-d6aab765554c" path="/var/lib/kubelet/pods/1111d626-023b-4ece-866e-d6aab765554c/volumes"
Sep 17 23:12:18.151225 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:18.151136    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c479e3-8a7c-4388-b45b-0483ff8781e6","Type":"ContainerStarted","Data":"9241ad7025b9549616e29fe7510548b4e2d1d2ca71720820407d591896a9527a"}
Sep 17 23:12:18.151225 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:18.151217    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c479e3-8a7c-4388-b45b-0483ff8781e6","Type":"ContainerStarted","Data":"fa79a8f82221732d7d66136862f09111448c1fc8f4c1eb472cb79a3cc2a4ea35"}
Sep 17 23:12:18.151649 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:18.151256    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c479e3-8a7c-4388-b45b-0483ff8781e6","Type":"ContainerStarted","Data":"3a3183d2bc89e86d717f878f9f11b14827250d4d74a8d4bb90ad8cd7cf670bf8"}
Sep 17 23:12:18.151649 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:18.151295    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c479e3-8a7c-4388-b45b-0483ff8781e6","Type":"ContainerStarted","Data":"905dd2c77a0732a4ef62c4932e29cb6ed5ed65dff1dc3ece2e001096e5babcf2"}
Sep 17 23:12:18.151649 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:18.151331    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c479e3-8a7c-4388-b45b-0483ff8781e6","Type":"ContainerStarted","Data":"fb78922801a63fda19cae11f135f757f98ae7863b6563f94119f58d823ec09e7"}
Sep 17 23:12:18.151649 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:18.151366    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c479e3-8a7c-4388-b45b-0483ff8781e6","Type":"ContainerStarted","Data":"40764145e30d3b66c93e592955bb07850c0451bd216433e849a07c26ccd28fae"}
Sep 17 23:12:18.185802 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:18.185752    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.170724366 podStartE2EDuration="2.185735739s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.011373ms" isStatefulPod=false podCreationTimestamp="2026-09-17 23:12:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:12:18.183966081 +0000 UTC m=+362.364604518" watchObservedRunningTime="2026-09-17 23:12:18.185735739 +0000 UTC m=+362.366374210"
Sep 17 23:12:20.024700 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:20.024667    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:21.769348 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:21.769315    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b55c867cd-p86bz"]
Sep 17 23:12:21.880220 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:21.880193    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:21.982159 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:21.982130    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:22.046034 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:22.045954    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:22.288261 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:22.288232    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:22.390247 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:22.390174    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:34.052687 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:34.052650    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:34.831904 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:34.831873    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:34.907552 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:34.907510    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:12:46.787952 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:46.787912    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b55c867cd-p86bz" podUID="5a99f446-1c86-438f-abd9-4063d8f813bc" containerName="console" containerID="cri-o://006fe630612396ca38e94ab1e6b531c1c168c56c45494ca592a50ee7deab5cc4" gracePeriod=15
Sep 17 23:12:47.023084 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.023060    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:12:47.101451 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.101363    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-trusted-ca-bundle\") pod \"5a99f446-1c86-438f-abd9-4063d8f813bc\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") "
Sep 17 23:12:47.101451 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.101405    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-console-config\") pod \"5a99f446-1c86-438f-abd9-4063d8f813bc\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") "
Sep 17 23:12:47.101451 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.101422    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47p8m\" (UniqueName: \"kubernetes.io/projected/5a99f446-1c86-438f-abd9-4063d8f813bc-kube-api-access-47p8m\") pod \"5a99f446-1c86-438f-abd9-4063d8f813bc\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") "
Sep 17 23:12:47.101451 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.101440    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a99f446-1c86-438f-abd9-4063d8f813bc-console-serving-cert\") pod \"5a99f446-1c86-438f-abd9-4063d8f813bc\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") "
Sep 17 23:12:47.101783 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.101467    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a99f446-1c86-438f-abd9-4063d8f813bc-console-oauth-config\") pod \"5a99f446-1c86-438f-abd9-4063d8f813bc\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") "
Sep 17 23:12:47.101783 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.101505    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-service-ca\") pod \"5a99f446-1c86-438f-abd9-4063d8f813bc\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") "
Sep 17 23:12:47.101783 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.101548    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-oauth-serving-cert\") pod \"5a99f446-1c86-438f-abd9-4063d8f813bc\" (UID: \"5a99f446-1c86-438f-abd9-4063d8f813bc\") "
Sep 17 23:12:47.102022 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.101990    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5a99f446-1c86-438f-abd9-4063d8f813bc" (UID: "5a99f446-1c86-438f-abd9-4063d8f813bc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:12:47.102022 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.102013    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-console-config" (OuterVolumeSpecName: "console-config") pod "5a99f446-1c86-438f-abd9-4063d8f813bc" (UID: "5a99f446-1c86-438f-abd9-4063d8f813bc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:12:47.102173 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.102054    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-service-ca" (OuterVolumeSpecName: "service-ca") pod "5a99f446-1c86-438f-abd9-4063d8f813bc" (UID: "5a99f446-1c86-438f-abd9-4063d8f813bc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:12:47.102173 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.102076    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5a99f446-1c86-438f-abd9-4063d8f813bc" (UID: "5a99f446-1c86-438f-abd9-4063d8f813bc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:12:47.104001 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.103977    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a99f446-1c86-438f-abd9-4063d8f813bc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5a99f446-1c86-438f-abd9-4063d8f813bc" (UID: "5a99f446-1c86-438f-abd9-4063d8f813bc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:12:47.104086 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.103994    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a99f446-1c86-438f-abd9-4063d8f813bc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5a99f446-1c86-438f-abd9-4063d8f813bc" (UID: "5a99f446-1c86-438f-abd9-4063d8f813bc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:12:47.104086 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.104002    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a99f446-1c86-438f-abd9-4063d8f813bc-kube-api-access-47p8m" (OuterVolumeSpecName: "kube-api-access-47p8m") pod "5a99f446-1c86-438f-abd9-4063d8f813bc" (UID: "5a99f446-1c86-438f-abd9-4063d8f813bc"). InnerVolumeSpecName "kube-api-access-47p8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:12:47.202568 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.202504    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-trusted-ca-bundle\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:47.202568 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.202556    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-console-config\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:47.202568 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.202572    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47p8m\" (UniqueName: \"kubernetes.io/projected/5a99f446-1c86-438f-abd9-4063d8f813bc-kube-api-access-47p8m\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:47.202800 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.202590    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a99f446-1c86-438f-abd9-4063d8f813bc-console-serving-cert\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:47.202800 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.202603    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a99f446-1c86-438f-abd9-4063d8f813bc-console-oauth-config\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:47.202800 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.202611    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:47.202800 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.202620    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a99f446-1c86-438f-abd9-4063d8f813bc-oauth-serving-cert\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:12:47.253670 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.253636    2558 generic.go:356] "Generic (PLEG): container finished" podID="5a99f446-1c86-438f-abd9-4063d8f813bc" containerID="006fe630612396ca38e94ab1e6b531c1c168c56c45494ca592a50ee7deab5cc4" exitCode=0
Sep 17 23:12:47.253818 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.253683    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b55c867cd-p86bz"
Sep 17 23:12:47.253818 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.253705    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b55c867cd-p86bz" event={"ID":"5a99f446-1c86-438f-abd9-4063d8f813bc","Type":"ContainerDied","Data":"006fe630612396ca38e94ab1e6b531c1c168c56c45494ca592a50ee7deab5cc4"}
Sep 17 23:12:47.253818 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.253743    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b55c867cd-p86bz" event={"ID":"5a99f446-1c86-438f-abd9-4063d8f813bc","Type":"ContainerDied","Data":"1e21d341556de259b18bd556e5aeaa84a6b563217ef31f7225e349209b69a1b7"}
Sep 17 23:12:47.253818 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.253762    2558 scope.go:117] "RemoveContainer" containerID="006fe630612396ca38e94ab1e6b531c1c168c56c45494ca592a50ee7deab5cc4"
Sep 17 23:12:47.262495 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.262479    2558 scope.go:117] "RemoveContainer" containerID="006fe630612396ca38e94ab1e6b531c1c168c56c45494ca592a50ee7deab5cc4"
Sep 17 23:12:47.262784 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:12:47.262765    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"006fe630612396ca38e94ab1e6b531c1c168c56c45494ca592a50ee7deab5cc4\": container with ID starting with 006fe630612396ca38e94ab1e6b531c1c168c56c45494ca592a50ee7deab5cc4 not found: ID does not exist" containerID="006fe630612396ca38e94ab1e6b531c1c168c56c45494ca592a50ee7deab5cc4"
Sep 17 23:12:47.262826 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.262793    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"006fe630612396ca38e94ab1e6b531c1c168c56c45494ca592a50ee7deab5cc4"} err="failed to get container status \"006fe630612396ca38e94ab1e6b531c1c168c56c45494ca592a50ee7deab5cc4\": rpc error: code = NotFound desc = could not find container \"006fe630612396ca38e94ab1e6b531c1c168c56c45494ca592a50ee7deab5cc4\": container with ID starting with 006fe630612396ca38e94ab1e6b531c1c168c56c45494ca592a50ee7deab5cc4 not found: ID does not exist"
Sep 17 23:12:47.274011 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.273988    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b55c867cd-p86bz"]
Sep 17 23:12:47.276984 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.276962    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b55c867cd-p86bz"]
Sep 17 23:12:47.930225 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:12:47.930189    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a99f446-1c86-438f-abd9-4063d8f813bc" path="/var/lib/kubelet/pods/5a99f446-1c86-438f-abd9-4063d8f813bc/volumes"
Sep 17 23:13:14.753894 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:14.753860    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:14.814530 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:14.814496    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:25.803117 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:25.803059    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:25.899266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:25.899238    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:26.460969 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.460938    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"]
Sep 17 23:13:26.461185 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.461174    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a99f446-1c86-438f-abd9-4063d8f813bc" containerName="console"
Sep 17 23:13:26.461230 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.461187    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a99f446-1c86-438f-abd9-4063d8f813bc" containerName="console"
Sep 17 23:13:26.461268 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.461244    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a99f446-1c86-438f-abd9-4063d8f813bc" containerName="console"
Sep 17 23:13:26.463733 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.463717    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"
Sep 17 23:13:26.466117 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.466098    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 23:13:26.466813 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.466796    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-86r94\""
Sep 17 23:13:26.466873 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.466796    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 23:13:26.474392 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.474374    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"]
Sep 17 23:13:26.477135 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.477111    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdnwt\" (UniqueName: \"kubernetes.io/projected/463b2338-f5b4-4564-a6c1-5f144d7d68bf-kube-api-access-gdnwt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9\" (UID: \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"
Sep 17 23:13:26.477226 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.477148    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/463b2338-f5b4-4564-a6c1-5f144d7d68bf-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9\" (UID: \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"
Sep 17 23:13:26.477226 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.477165    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/463b2338-f5b4-4564-a6c1-5f144d7d68bf-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9\" (UID: \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"
Sep 17 23:13:26.578099 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.578069    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/463b2338-f5b4-4564-a6c1-5f144d7d68bf-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9\" (UID: \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"
Sep 17 23:13:26.578245 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.578129    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdnwt\" (UniqueName: \"kubernetes.io/projected/463b2338-f5b4-4564-a6c1-5f144d7d68bf-kube-api-access-gdnwt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9\" (UID: \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"
Sep 17 23:13:26.578245 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.578164    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/463b2338-f5b4-4564-a6c1-5f144d7d68bf-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9\" (UID: \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"
Sep 17 23:13:26.578434 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.578414    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/463b2338-f5b4-4564-a6c1-5f144d7d68bf-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9\" (UID: \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"
Sep 17 23:13:26.578509 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.578487    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/463b2338-f5b4-4564-a6c1-5f144d7d68bf-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9\" (UID: \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"
Sep 17 23:13:26.586671 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.586649    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdnwt\" (UniqueName: \"kubernetes.io/projected/463b2338-f5b4-4564-a6c1-5f144d7d68bf-kube-api-access-gdnwt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9\" (UID: \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"
Sep 17 23:13:26.772156 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:26.772100    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"
Sep 17 23:13:27.040350 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:27.040327    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"]
Sep 17 23:13:27.042957 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:13:27.042926    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod463b2338_f5b4_4564_a6c1_5f144d7d68bf.slice/crio-921ea222574e3d38a4652376058e626a55274ad6e50faeb9a975922162924f32 WatchSource:0}: Error finding container 921ea222574e3d38a4652376058e626a55274ad6e50faeb9a975922162924f32: Status 404 returned error can't find the container with id 921ea222574e3d38a4652376058e626a55274ad6e50faeb9a975922162924f32
Sep 17 23:13:27.045238 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:27.045220    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:13:27.389251 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:27.389195    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9" event={"ID":"463b2338-f5b4-4564-a6c1-5f144d7d68bf","Type":"ContainerStarted","Data":"921ea222574e3d38a4652376058e626a55274ad6e50faeb9a975922162924f32"}
Sep 17 23:13:31.337649 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:31.337560    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:33.414823 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:33.414793    2558 generic.go:356] "Generic (PLEG): container finished" podID="463b2338-f5b4-4564-a6c1-5f144d7d68bf" containerID="e0cdf09ae4c42e7b130e9fcd910318887e137af11e1def0648a49dbf9b37cc40" exitCode=0
Sep 17 23:13:33.415152 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:33.414846    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9" event={"ID":"463b2338-f5b4-4564-a6c1-5f144d7d68bf","Type":"ContainerDied","Data":"e0cdf09ae4c42e7b130e9fcd910318887e137af11e1def0648a49dbf9b37cc40"}
Sep 17 23:13:35.424190 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:35.424163    2558 generic.go:356] "Generic (PLEG): container finished" podID="463b2338-f5b4-4564-a6c1-5f144d7d68bf" containerID="d0c1627da5b81c25205000b36d41a7ada42c861a3dff4826b501edb512cfe1ea" exitCode=0
Sep 17 23:13:35.424536 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:35.424248    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9" event={"ID":"463b2338-f5b4-4564-a6c1-5f144d7d68bf","Type":"ContainerDied","Data":"d0c1627da5b81c25205000b36d41a7ada42c861a3dff4826b501edb512cfe1ea"}
Sep 17 23:13:43.454495 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:43.454463    2558 generic.go:356] "Generic (PLEG): container finished" podID="463b2338-f5b4-4564-a6c1-5f144d7d68bf" containerID="bf7072e7ab16ff8881e57a235adc73f279b1b3fa26ead6ed4c766213af9f5fa9" exitCode=0
Sep 17 23:13:43.454883 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:43.454554    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9" event={"ID":"463b2338-f5b4-4564-a6c1-5f144d7d68bf","Type":"ContainerDied","Data":"bf7072e7ab16ff8881e57a235adc73f279b1b3fa26ead6ed4c766213af9f5fa9"}
Sep 17 23:13:44.581649 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:44.581628    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"
Sep 17 23:13:44.705448 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:44.705422    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/463b2338-f5b4-4564-a6c1-5f144d7d68bf-util\") pod \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\" (UID: \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\") "
Sep 17 23:13:44.705594 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:44.705468    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdnwt\" (UniqueName: \"kubernetes.io/projected/463b2338-f5b4-4564-a6c1-5f144d7d68bf-kube-api-access-gdnwt\") pod \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\" (UID: \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\") "
Sep 17 23:13:44.705594 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:44.705493    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/463b2338-f5b4-4564-a6c1-5f144d7d68bf-bundle\") pod \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\" (UID: \"463b2338-f5b4-4564-a6c1-5f144d7d68bf\") "
Sep 17 23:13:44.706111 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:44.706087    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/463b2338-f5b4-4564-a6c1-5f144d7d68bf-bundle" (OuterVolumeSpecName: "bundle") pod "463b2338-f5b4-4564-a6c1-5f144d7d68bf" (UID: "463b2338-f5b4-4564-a6c1-5f144d7d68bf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:13:44.707452 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:44.707432    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/463b2338-f5b4-4564-a6c1-5f144d7d68bf-kube-api-access-gdnwt" (OuterVolumeSpecName: "kube-api-access-gdnwt") pod "463b2338-f5b4-4564-a6c1-5f144d7d68bf" (UID: "463b2338-f5b4-4564-a6c1-5f144d7d68bf"). InnerVolumeSpecName "kube-api-access-gdnwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:13:44.710597 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:44.710548    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/463b2338-f5b4-4564-a6c1-5f144d7d68bf-util" (OuterVolumeSpecName: "util") pod "463b2338-f5b4-4564-a6c1-5f144d7d68bf" (UID: "463b2338-f5b4-4564-a6c1-5f144d7d68bf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:13:44.806086 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:44.806058    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/463b2338-f5b4-4564-a6c1-5f144d7d68bf-util\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:13:44.806086 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:44.806083    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdnwt\" (UniqueName: \"kubernetes.io/projected/463b2338-f5b4-4564-a6c1-5f144d7d68bf-kube-api-access-gdnwt\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:13:44.806226 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:44.806097    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/463b2338-f5b4-4564-a6c1-5f144d7d68bf-bundle\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:13:45.463907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:45.463870    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9" event={"ID":"463b2338-f5b4-4564-a6c1-5f144d7d68bf","Type":"ContainerDied","Data":"921ea222574e3d38a4652376058e626a55274ad6e50faeb9a975922162924f32"}
Sep 17 23:13:45.464068 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:45.463923    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="921ea222574e3d38a4652376058e626a55274ad6e50faeb9a975922162924f32"
Sep 17 23:13:45.464068 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:45.463889    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409cnl9"
Sep 17 23:13:47.441730 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:47.441705    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:49.278193 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.278161    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:49.440579 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.440551    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f"]
Sep 17 23:13:49.440786 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.440774    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="463b2338-f5b4-4564-a6c1-5f144d7d68bf" containerName="extract"
Sep 17 23:13:49.440786 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.440786    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="463b2338-f5b4-4564-a6c1-5f144d7d68bf" containerName="extract"
Sep 17 23:13:49.440870 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.440805    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="463b2338-f5b4-4564-a6c1-5f144d7d68bf" containerName="util"
Sep 17 23:13:49.440870 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.440810    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="463b2338-f5b4-4564-a6c1-5f144d7d68bf" containerName="util"
Sep 17 23:13:49.440870 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.440817    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="463b2338-f5b4-4564-a6c1-5f144d7d68bf" containerName="pull"
Sep 17 23:13:49.440870 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.440823    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="463b2338-f5b4-4564-a6c1-5f144d7d68bf" containerName="pull"
Sep 17 23:13:49.440870 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.440866    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="463b2338-f5b4-4564-a6c1-5f144d7d68bf" containerName="extract"
Sep 17 23:13:49.496104 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.496081    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f"]
Sep 17 23:13:49.496224 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.496158    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f"
Sep 17 23:13:49.498807 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.498790    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 17 23:13:49.498907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.498804    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 17 23:13:49.498907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.498793    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-tbzdk\""
Sep 17 23:13:49.536554 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.536494    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x6rf\" (UniqueName: \"kubernetes.io/projected/49f91a23-6273-4124-984c-73c3da3bed13-kube-api-access-8x6rf\") pod \"cert-manager-operator-controller-manager-55c444bf86-xgq2f\" (UID: \"49f91a23-6273-4124-984c-73c3da3bed13\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f"
Sep 17 23:13:49.536633 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.536593    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49f91a23-6273-4124-984c-73c3da3bed13-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-xgq2f\" (UID: \"49f91a23-6273-4124-984c-73c3da3bed13\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f"
Sep 17 23:13:49.637827 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.637803    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49f91a23-6273-4124-984c-73c3da3bed13-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-xgq2f\" (UID: \"49f91a23-6273-4124-984c-73c3da3bed13\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f"
Sep 17 23:13:49.637895 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.637841    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x6rf\" (UniqueName: \"kubernetes.io/projected/49f91a23-6273-4124-984c-73c3da3bed13-kube-api-access-8x6rf\") pod \"cert-manager-operator-controller-manager-55c444bf86-xgq2f\" (UID: \"49f91a23-6273-4124-984c-73c3da3bed13\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f"
Sep 17 23:13:49.638130 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.638112    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49f91a23-6273-4124-984c-73c3da3bed13-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-xgq2f\" (UID: \"49f91a23-6273-4124-984c-73c3da3bed13\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f"
Sep 17 23:13:49.646129 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.646103    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x6rf\" (UniqueName: \"kubernetes.io/projected/49f91a23-6273-4124-984c-73c3da3bed13-kube-api-access-8x6rf\") pod \"cert-manager-operator-controller-manager-55c444bf86-xgq2f\" (UID: \"49f91a23-6273-4124-984c-73c3da3bed13\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f"
Sep 17 23:13:49.805832 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:49.805788    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f"
Sep 17 23:13:50.108756 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:50.108734    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f"]
Sep 17 23:13:50.111479 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:13:50.111455    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49f91a23_6273_4124_984c_73c3da3bed13.slice/crio-e04b57b45e916285d7fea9c3437cd34210016970899191dce549f303db6b6de1 WatchSource:0}: Error finding container e04b57b45e916285d7fea9c3437cd34210016970899191dce549f303db6b6de1: Status 404 returned error can't find the container with id e04b57b45e916285d7fea9c3437cd34210016970899191dce549f303db6b6de1
Sep 17 23:13:50.484386 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:50.484352    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f" event={"ID":"49f91a23-6273-4124-984c-73c3da3bed13","Type":"ContainerStarted","Data":"e04b57b45e916285d7fea9c3437cd34210016970899191dce549f303db6b6de1"}
Sep 17 23:13:51.101626 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:51.101592    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:51.201246 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:51.201218    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:52.495966 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:52.495887    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f" event={"ID":"49f91a23-6273-4124-984c-73c3da3bed13","Type":"ContainerStarted","Data":"08bff6f85bb003ca8058c4a118af321af5ecd04d45ea6d4b50fd29eff6eae5f2"}
Sep 17 23:13:52.515638 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:52.515597    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-xgq2f" podStartSLOduration=1.438565031 podStartE2EDuration="3.515584075s" totalImagesPullingTime="2.077019044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:13:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:13:52.515281583 +0000 UTC m=+456.695920020" watchObservedRunningTime="2026-09-17 23:13:52.515584075 +0000 UTC m=+456.696222513"
Sep 17 23:13:52.917257 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:52.917178    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:53.061262 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:53.061236    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:54.270980 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.270951    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"]
Sep 17 23:13:54.285675 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.285646    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"]
Sep 17 23:13:54.285821 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.285772    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"
Sep 17 23:13:54.288475 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.288451    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 23:13:54.288616 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.288488    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 23:13:54.289402 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.289380    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-86r94\""
Sep 17 23:13:54.372185 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.372165    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58\" (UID: \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"
Sep 17 23:13:54.372289 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.372190    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58\" (UID: \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"
Sep 17 23:13:54.372289 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.372211    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzfqr\" (UniqueName: \"kubernetes.io/projected/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-kube-api-access-lzfqr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58\" (UID: \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"
Sep 17 23:13:54.472672 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.472646    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58\" (UID: \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"
Sep 17 23:13:54.472672 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.472673    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58\" (UID: \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"
Sep 17 23:13:54.472791 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.472693    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzfqr\" (UniqueName: \"kubernetes.io/projected/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-kube-api-access-lzfqr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58\" (UID: \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"
Sep 17 23:13:54.472978 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.472963    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58\" (UID: \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"
Sep 17 23:13:54.473013 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.472989    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58\" (UID: \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"
Sep 17 23:13:54.480407 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.480379    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzfqr\" (UniqueName: \"kubernetes.io/projected/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-kube-api-access-lzfqr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58\" (UID: \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"
Sep 17 23:13:54.595104 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.595056    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"
Sep 17 23:13:54.884271 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:54.884196    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"]
Sep 17 23:13:54.886874 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:13:54.886848    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11911a34_fb9e_4b1e_bb62_aa33d4bb2a1f.slice/crio-88174333fb71d829cfe88b8e866aa1a43990ec86300525644268a878614c95be WatchSource:0}: Error finding container 88174333fb71d829cfe88b8e866aa1a43990ec86300525644268a878614c95be: Status 404 returned error can't find the container with id 88174333fb71d829cfe88b8e866aa1a43990ec86300525644268a878614c95be
Sep 17 23:13:55.261815 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:55.261787    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:55.507427 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:55.507400    2558 generic.go:356] "Generic (PLEG): container finished" podID="11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f" containerID="3827f27345d684dc85f1027f3cf47c302c6079bef4b065e8300ccd2ab324d679" exitCode=0
Sep 17 23:13:55.507732 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:55.507487    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58" event={"ID":"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f","Type":"ContainerDied","Data":"3827f27345d684dc85f1027f3cf47c302c6079bef4b065e8300ccd2ab324d679"}
Sep 17 23:13:55.507732 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:55.507561    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58" event={"ID":"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f","Type":"ContainerStarted","Data":"88174333fb71d829cfe88b8e866aa1a43990ec86300525644268a878614c95be"}
Sep 17 23:13:55.677179 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:55.677116    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:57.517748 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:57.517722    2558 generic.go:356] "Generic (PLEG): container finished" podID="11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f" containerID="e218006ac10a9443eeff6c3498410c8cd8553c8b96315e27925cd5da5106f453" exitCode=0
Sep 17 23:13:57.518036 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:57.517781    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58" event={"ID":"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f","Type":"ContainerDied","Data":"e218006ac10a9443eeff6c3498410c8cd8553c8b96315e27925cd5da5106f453"}
Sep 17 23:13:57.843140 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:57.843113    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:57.896126 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:57.896074    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:13:58.524075 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:58.524041    2558 generic.go:356] "Generic (PLEG): container finished" podID="11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f" containerID="1f87895c079a6cfa09b2c7e1ef2c30bd65286f226967803846819b6de8d542d4" exitCode=0
Sep 17 23:13:58.524498 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:58.524110    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58" event={"ID":"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f","Type":"ContainerDied","Data":"1f87895c079a6cfa09b2c7e1ef2c30bd65286f226967803846819b6de8d542d4"}
Sep 17 23:13:59.645194 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:59.645171    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"
Sep 17 23:13:59.717313 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:59.717278    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-bundle\") pod \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\" (UID: \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\") "
Sep 17 23:13:59.717313 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:59.717318    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-util\") pod \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\" (UID: \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\") "
Sep 17 23:13:59.717509 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:59.717368    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzfqr\" (UniqueName: \"kubernetes.io/projected/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-kube-api-access-lzfqr\") pod \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\" (UID: \"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f\") "
Sep 17 23:13:59.717719 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:59.717696    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-bundle" (OuterVolumeSpecName: "bundle") pod "11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f" (UID: "11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:13:59.719310 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:59.719273    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-kube-api-access-lzfqr" (OuterVolumeSpecName: "kube-api-access-lzfqr") pod "11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f" (UID: "11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f"). InnerVolumeSpecName "kube-api-access-lzfqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:13:59.722742 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:59.722721    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-util" (OuterVolumeSpecName: "util") pod "11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f" (UID: "11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:13:59.818566 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:59.818503    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzfqr\" (UniqueName: \"kubernetes.io/projected/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-kube-api-access-lzfqr\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:13:59.818566 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:59.818539    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-bundle\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:13:59.818566 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:13:59.818548    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f-util\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:14:00.533004 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:00.532977    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58"
Sep 17 23:14:00.533158 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:00.532977    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftgs58" event={"ID":"11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f","Type":"ContainerDied","Data":"88174333fb71d829cfe88b8e866aa1a43990ec86300525644268a878614c95be"}
Sep 17 23:14:00.533158 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:00.533096    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88174333fb71d829cfe88b8e866aa1a43990ec86300525644268a878614c95be"
Sep 17 23:14:02.667085 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:02.667056    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:03.553526 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.553480    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wzmzn"]
Sep 17 23:14:03.553768 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.553755    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f" containerName="util"
Sep 17 23:14:03.553812 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.553770    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f" containerName="util"
Sep 17 23:14:03.553812 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.553780    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f" containerName="extract"
Sep 17 23:14:03.553812 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.553785    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f" containerName="extract"
Sep 17 23:14:03.553812 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.553798    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f" containerName="pull"
Sep 17 23:14:03.553812 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.553804    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f" containerName="pull"
Sep 17 23:14:03.553959 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.553857    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="11911a34-fb9e-4b1e-bb62-aa33d4bb2a1f" containerName="extract"
Sep 17 23:14:03.555435 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.555419    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wzmzn"
Sep 17 23:14:03.557661 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.557639    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 17 23:14:03.557798 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.557665    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 17 23:14:03.558465 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.558445    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-lst9r\""
Sep 17 23:14:03.566737 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.566714    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wzmzn"]
Sep 17 23:14:03.643451 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.643424    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7571082d-09b5-46a6-8e58-6a4af1f25cf5-bound-sa-token\") pod \"cert-manager-55b4d954d5-wzmzn\" (UID: \"7571082d-09b5-46a6-8e58-6a4af1f25cf5\") " pod="cert-manager/cert-manager-55b4d954d5-wzmzn"
Sep 17 23:14:03.643563 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.643460    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcl7h\" (UniqueName: \"kubernetes.io/projected/7571082d-09b5-46a6-8e58-6a4af1f25cf5-kube-api-access-qcl7h\") pod \"cert-manager-55b4d954d5-wzmzn\" (UID: \"7571082d-09b5-46a6-8e58-6a4af1f25cf5\") " pod="cert-manager/cert-manager-55b4d954d5-wzmzn"
Sep 17 23:14:03.708309 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.708278    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:03.744440 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.744415    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7571082d-09b5-46a6-8e58-6a4af1f25cf5-bound-sa-token\") pod \"cert-manager-55b4d954d5-wzmzn\" (UID: \"7571082d-09b5-46a6-8e58-6a4af1f25cf5\") " pod="cert-manager/cert-manager-55b4d954d5-wzmzn"
Sep 17 23:14:03.744570 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.744450    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcl7h\" (UniqueName: \"kubernetes.io/projected/7571082d-09b5-46a6-8e58-6a4af1f25cf5-kube-api-access-qcl7h\") pod \"cert-manager-55b4d954d5-wzmzn\" (UID: \"7571082d-09b5-46a6-8e58-6a4af1f25cf5\") " pod="cert-manager/cert-manager-55b4d954d5-wzmzn"
Sep 17 23:14:03.751830 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.751805    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7571082d-09b5-46a6-8e58-6a4af1f25cf5-bound-sa-token\") pod \"cert-manager-55b4d954d5-wzmzn\" (UID: \"7571082d-09b5-46a6-8e58-6a4af1f25cf5\") " pod="cert-manager/cert-manager-55b4d954d5-wzmzn"
Sep 17 23:14:03.751922 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.751867    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcl7h\" (UniqueName: \"kubernetes.io/projected/7571082d-09b5-46a6-8e58-6a4af1f25cf5-kube-api-access-qcl7h\") pod \"cert-manager-55b4d954d5-wzmzn\" (UID: \"7571082d-09b5-46a6-8e58-6a4af1f25cf5\") " pod="cert-manager/cert-manager-55b4d954d5-wzmzn"
Sep 17 23:14:03.763204 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.763182    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:03.864408 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:03.864330    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wzmzn"
Sep 17 23:14:04.129802 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:04.129780    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wzmzn"]
Sep 17 23:14:04.132255 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:14:04.132231    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7571082d_09b5_46a6_8e58_6a4af1f25cf5.slice/crio-da1fe7144b41be7b19ad40ecf52f82e1c9ce057671db62797a9e9ee9abccdfcd WatchSource:0}: Error finding container da1fe7144b41be7b19ad40ecf52f82e1c9ce057671db62797a9e9ee9abccdfcd: Status 404 returned error can't find the container with id da1fe7144b41be7b19ad40ecf52f82e1c9ce057671db62797a9e9ee9abccdfcd
Sep 17 23:14:04.262370 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:04.262347    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:04.361531 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:04.361490    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:04.548956 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:04.548924    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wzmzn" event={"ID":"7571082d-09b5-46a6-8e58-6a4af1f25cf5","Type":"ContainerStarted","Data":"da1fe7144b41be7b19ad40ecf52f82e1c9ce057671db62797a9e9ee9abccdfcd"}
Sep 17 23:14:07.567668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:07.567627    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wzmzn" event={"ID":"7571082d-09b5-46a6-8e58-6a4af1f25cf5","Type":"ContainerStarted","Data":"756b4d099b7d5bd70648340ad9be6f6835050deb7e616f9aa56e55cd175cb972"}
Sep 17 23:14:07.584560 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:07.584498    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-wzmzn" podStartSLOduration=1.966366192 podStartE2EDuration="4.584484626s" totalImagesPullingTime="2.618118434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:14:07.58329692 +0000 UTC m=+471.763935356" watchObservedRunningTime="2026-09-17 23:14:07.584484626 +0000 UTC m=+471.765123064"
Sep 17 23:14:10.192227 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:10.192184    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:10.293570 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:10.293538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:12.652442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:12.652415    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:14.055573 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.052038    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"]
Sep 17 23:14:14.055573 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.055504    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"
Sep 17 23:14:14.058081 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.058059    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 23:14:14.058224 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.058111    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-86r94\""
Sep 17 23:14:14.058913 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.058900    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 23:14:14.061538 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.061502    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"]
Sep 17 23:14:14.115686 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.115666    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm\" (UID: \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"
Sep 17 23:14:14.115829 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.115696    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx2lm\" (UniqueName: \"kubernetes.io/projected/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-kube-api-access-jx2lm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm\" (UID: \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"
Sep 17 23:14:14.115829 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.115717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm\" (UID: \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"
Sep 17 23:14:14.216225 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.216202    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm\" (UID: \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"
Sep 17 23:14:14.216312 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.216232    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx2lm\" (UniqueName: \"kubernetes.io/projected/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-kube-api-access-jx2lm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm\" (UID: \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"
Sep 17 23:14:14.216312 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.216253    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm\" (UID: \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"
Sep 17 23:14:14.216546 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.216526    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm\" (UID: \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"
Sep 17 23:14:14.216639 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.216624    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm\" (UID: \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"
Sep 17 23:14:14.224677 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.224653    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx2lm\" (UniqueName: \"kubernetes.io/projected/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-kube-api-access-jx2lm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm\" (UID: \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"
Sep 17 23:14:14.365503 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.365456    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"
Sep 17 23:14:14.634641 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:14.634541    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"]
Sep 17 23:14:14.637952 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:14:14.637926    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e2abcc3_e377_4d13_b4e4_be07bbfb0875.slice/crio-67e4146865f59d40603511c533e5c1fec968b3372064002831a3518319e2cbd5 WatchSource:0}: Error finding container 67e4146865f59d40603511c533e5c1fec968b3372064002831a3518319e2cbd5: Status 404 returned error can't find the container with id 67e4146865f59d40603511c533e5c1fec968b3372064002831a3518319e2cbd5
Sep 17 23:14:15.601294 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:15.601226    2558 generic.go:356] "Generic (PLEG): container finished" podID="4e2abcc3-e377-4d13-b4e4-be07bbfb0875" containerID="76c6d7c9644504aecd6c24b278a58d2a8c99ee8f98c195cb838629e86c7f456a" exitCode=0
Sep 17 23:14:15.601294 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:15.601287    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm" event={"ID":"4e2abcc3-e377-4d13-b4e4-be07bbfb0875","Type":"ContainerDied","Data":"76c6d7c9644504aecd6c24b278a58d2a8c99ee8f98c195cb838629e86c7f456a"}
Sep 17 23:14:15.601667 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:15.601327    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm" event={"ID":"4e2abcc3-e377-4d13-b4e4-be07bbfb0875","Type":"ContainerStarted","Data":"67e4146865f59d40603511c533e5c1fec968b3372064002831a3518319e2cbd5"}
Sep 17 23:14:16.606944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:16.606856    2558 generic.go:356] "Generic (PLEG): container finished" podID="4e2abcc3-e377-4d13-b4e4-be07bbfb0875" containerID="afb535fee850da4c9f19aa0510fab8300665381c9f7f25c2865fcfd5f4512ec8" exitCode=0
Sep 17 23:14:16.606944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:16.606890    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm" event={"ID":"4e2abcc3-e377-4d13-b4e4-be07bbfb0875","Type":"ContainerDied","Data":"afb535fee850da4c9f19aa0510fab8300665381c9f7f25c2865fcfd5f4512ec8"}
Sep 17 23:14:17.613509 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:17.613473    2558 generic.go:356] "Generic (PLEG): container finished" podID="4e2abcc3-e377-4d13-b4e4-be07bbfb0875" containerID="1d7639180dbf5a19af281579679cfa173ed7ab875bcab8ebd0f09b2a3cf88fe0" exitCode=0
Sep 17 23:14:17.613905 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:17.613569    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm" event={"ID":"4e2abcc3-e377-4d13-b4e4-be07bbfb0875","Type":"ContainerDied","Data":"1d7639180dbf5a19af281579679cfa173ed7ab875bcab8ebd0f09b2a3cf88fe0"}
Sep 17 23:14:18.732740 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:18.732715    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"
Sep 17 23:14:18.850447 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:18.850409    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-util\") pod \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\" (UID: \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\") "
Sep 17 23:14:18.850605 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:18.850473    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-bundle\") pod \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\" (UID: \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\") "
Sep 17 23:14:18.850605 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:18.850500    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jx2lm\" (UniqueName: \"kubernetes.io/projected/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-kube-api-access-jx2lm\") pod \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\" (UID: \"4e2abcc3-e377-4d13-b4e4-be07bbfb0875\") "
Sep 17 23:14:18.851182 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:18.851153    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-bundle" (OuterVolumeSpecName: "bundle") pod "4e2abcc3-e377-4d13-b4e4-be07bbfb0875" (UID: "4e2abcc3-e377-4d13-b4e4-be07bbfb0875"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:14:18.852599 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:18.852570    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-kube-api-access-jx2lm" (OuterVolumeSpecName: "kube-api-access-jx2lm") pod "4e2abcc3-e377-4d13-b4e4-be07bbfb0875" (UID: "4e2abcc3-e377-4d13-b4e4-be07bbfb0875"). InnerVolumeSpecName "kube-api-access-jx2lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:14:18.856188 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:18.856170    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-util" (OuterVolumeSpecName: "util") pod "4e2abcc3-e377-4d13-b4e4-be07bbfb0875" (UID: "4e2abcc3-e377-4d13-b4e4-be07bbfb0875"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:14:18.951055 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:18.951037    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jx2lm\" (UniqueName: \"kubernetes.io/projected/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-kube-api-access-jx2lm\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:14:18.951141 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:18.951056    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-util\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:14:18.951141 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:18.951065    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e2abcc3-e377-4d13-b4e4-be07bbfb0875-bundle\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:14:19.623734 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:19.623711    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm"
Sep 17 23:14:19.623734 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:19.623714    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr7rm" event={"ID":"4e2abcc3-e377-4d13-b4e4-be07bbfb0875","Type":"ContainerDied","Data":"67e4146865f59d40603511c533e5c1fec968b3372064002831a3518319e2cbd5"}
Sep 17 23:14:19.623937 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:19.623758    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67e4146865f59d40603511c533e5c1fec968b3372064002831a3518319e2cbd5"
Sep 17 23:14:25.068416 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.068387    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"]
Sep 17 23:14:25.068864 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.068634    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e2abcc3-e377-4d13-b4e4-be07bbfb0875" containerName="util"
Sep 17 23:14:25.068864 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.068645    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e2abcc3-e377-4d13-b4e4-be07bbfb0875" containerName="util"
Sep 17 23:14:25.068864 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.068654    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e2abcc3-e377-4d13-b4e4-be07bbfb0875" containerName="pull"
Sep 17 23:14:25.068864 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.068659    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e2abcc3-e377-4d13-b4e4-be07bbfb0875" containerName="pull"
Sep 17 23:14:25.068864 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.068671    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e2abcc3-e377-4d13-b4e4-be07bbfb0875" containerName="extract"
Sep 17 23:14:25.068864 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.068677    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e2abcc3-e377-4d13-b4e4-be07bbfb0875" containerName="extract"
Sep 17 23:14:25.068864 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.068719    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e2abcc3-e377-4d13-b4e4-be07bbfb0875" containerName="extract"
Sep 17 23:14:25.072449 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.072433    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"
Sep 17 23:14:25.074719 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.074695    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 23:14:25.074836 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.074777    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-86r94\""
Sep 17 23:14:25.075504 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.075490    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 23:14:25.084375 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.082949    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"]
Sep 17 23:14:25.193932 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.193899    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw\" (UID: \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"
Sep 17 23:14:25.194052 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.193966    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g27qs\" (UniqueName: \"kubernetes.io/projected/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-kube-api-access-g27qs\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw\" (UID: \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"
Sep 17 23:14:25.194052 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.194011    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw\" (UID: \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"
Sep 17 23:14:25.295163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.295141    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g27qs\" (UniqueName: \"kubernetes.io/projected/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-kube-api-access-g27qs\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw\" (UID: \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"
Sep 17 23:14:25.295251 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.295169    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw\" (UID: \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"
Sep 17 23:14:25.295251 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.295197    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw\" (UID: \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"
Sep 17 23:14:25.295477 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.295464    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw\" (UID: \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"
Sep 17 23:14:25.295536 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.295496    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw\" (UID: \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"
Sep 17 23:14:25.303632 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.303606    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g27qs\" (UniqueName: \"kubernetes.io/projected/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-kube-api-access-g27qs\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw\" (UID: \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"
Sep 17 23:14:25.380939 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.380894    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"
Sep 17 23:14:25.658302 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:25.658242    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"]
Sep 17 23:14:25.660866 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:14:25.660843    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b15067d_6b73_4fb4_a0cb_b1561c1fbd15.slice/crio-539a814de12f674e64500e699143bb8853f29a159e755bd164753d3449e3fcf1 WatchSource:0}: Error finding container 539a814de12f674e64500e699143bb8853f29a159e755bd164753d3449e3fcf1: Status 404 returned error can't find the container with id 539a814de12f674e64500e699143bb8853f29a159e755bd164753d3449e3fcf1
Sep 17 23:14:26.237106 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:26.237071    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:26.325789 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:26.325754    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:26.653140 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:26.653058    2558 generic.go:356] "Generic (PLEG): container finished" podID="7b15067d-6b73-4fb4-a0cb-b1561c1fbd15" containerID="9bb691ccf25e211c71f77f6f12408951826368d184576bc2b05a5fecf44e5864" exitCode=0
Sep 17 23:14:26.653140 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:26.653106    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw" event={"ID":"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15","Type":"ContainerDied","Data":"9bb691ccf25e211c71f77f6f12408951826368d184576bc2b05a5fecf44e5864"}
Sep 17 23:14:26.653316 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:26.653160    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw" event={"ID":"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15","Type":"ContainerStarted","Data":"539a814de12f674e64500e699143bb8853f29a159e755bd164753d3449e3fcf1"}
Sep 17 23:14:26.910008 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:26.909915    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:27.121347 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.121323    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:27.222527 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.222495    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:27.238265 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.238246    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"]
Sep 17 23:14:27.241282 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.241263    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.243943 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.243927    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 17 23:14:27.244727 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.244708    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-24qcm\""
Sep 17 23:14:27.244819 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.244724    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 17 23:14:27.244819 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.244706    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 17 23:14:27.244819 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.244779    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 17 23:14:27.256348 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.256331    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"]
Sep 17 23:14:27.369999 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.369977    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:27.409045 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.409021    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/253192e6-6970-47fe-bba9-ab238e390a33-manifests\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.409140 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.409051    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnhl8\" (UniqueName: \"kubernetes.io/projected/253192e6-6970-47fe-bba9-ab238e390a33-kube-api-access-xnhl8\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.409140 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.409076    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/253192e6-6970-47fe-bba9-ab238e390a33-tmp\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.409211 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.409134    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/253192e6-6970-47fe-bba9-ab238e390a33-apiservice-cert\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.409211 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.409171    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/253192e6-6970-47fe-bba9-ab238e390a33-webhook-cert\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.470000 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.469978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:27.510291 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.510265    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/253192e6-6970-47fe-bba9-ab238e390a33-apiservice-cert\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.510390 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.510298    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/253192e6-6970-47fe-bba9-ab238e390a33-webhook-cert\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.510390 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.510330    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/253192e6-6970-47fe-bba9-ab238e390a33-manifests\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.510545 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.510501    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnhl8\" (UniqueName: \"kubernetes.io/projected/253192e6-6970-47fe-bba9-ab238e390a33-kube-api-access-xnhl8\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.510634 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.510586    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/253192e6-6970-47fe-bba9-ab238e390a33-tmp\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.510727 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.510709    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/253192e6-6970-47fe-bba9-ab238e390a33-manifests\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.512799 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.512773    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/253192e6-6970-47fe-bba9-ab238e390a33-tmp\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.512905 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.512809    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/253192e6-6970-47fe-bba9-ab238e390a33-webhook-cert\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.512905 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.512861    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/253192e6-6970-47fe-bba9-ab238e390a33-apiservice-cert\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.517814 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.517793    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnhl8\" (UniqueName: \"kubernetes.io/projected/253192e6-6970-47fe-bba9-ab238e390a33-kube-api-access-xnhl8\") pod \"opendatahub-operator-controller-manager-747fdf4bf5-t85kq\" (UID: \"253192e6-6970-47fe-bba9-ab238e390a33\") " pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.560798 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.560778    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:27.578208 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.578185    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:27.662728 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.662703    2558 generic.go:356] "Generic (PLEG): container finished" podID="7b15067d-6b73-4fb4-a0cb-b1561c1fbd15" containerID="c902a14096a0edf48ec0c73d6ce8a255de24ede8170ec30664e7af18b9e80947" exitCode=0
Sep 17 23:14:27.662829 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.662745    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw" event={"ID":"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15","Type":"ContainerDied","Data":"c902a14096a0edf48ec0c73d6ce8a255de24ede8170ec30664e7af18b9e80947"}
Sep 17 23:14:27.676950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.676923    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:27.771109 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.771044    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:27.837000 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:27.836978    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"]
Sep 17 23:14:27.864248 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:14:27.864222    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod253192e6_6970_47fe_bba9_ab238e390a33.slice/crio-482611c1eef62bd14ad53ad74cab8dba4098ed4f54d97dc3e918203d05795d08 WatchSource:0}: Error finding container 482611c1eef62bd14ad53ad74cab8dba4098ed4f54d97dc3e918203d05795d08: Status 404 returned error can't find the container with id 482611c1eef62bd14ad53ad74cab8dba4098ed4f54d97dc3e918203d05795d08
Sep 17 23:14:28.670084 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:28.670043    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq" event={"ID":"253192e6-6970-47fe-bba9-ab238e390a33","Type":"ContainerStarted","Data":"482611c1eef62bd14ad53ad74cab8dba4098ed4f54d97dc3e918203d05795d08"}
Sep 17 23:14:28.672873 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:28.672833    2558 generic.go:356] "Generic (PLEG): container finished" podID="7b15067d-6b73-4fb4-a0cb-b1561c1fbd15" containerID="3af2ab44445e8c47ffe5c9d86d2f62877392eff2582f4350f1d92208e3630f4c" exitCode=0
Sep 17 23:14:28.673005 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:28.672891    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw" event={"ID":"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15","Type":"ContainerDied","Data":"3af2ab44445e8c47ffe5c9d86d2f62877392eff2582f4350f1d92208e3630f4c"}
Sep 17 23:14:30.802691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:30.802662    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"
Sep 17 23:14:30.937259 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:30.937234    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-util\") pod \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\" (UID: \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\") "
Sep 17 23:14:30.937351 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:30.937267    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g27qs\" (UniqueName: \"kubernetes.io/projected/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-kube-api-access-g27qs\") pod \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\" (UID: \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\") "
Sep 17 23:14:30.937351 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:30.937289    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-bundle\") pod \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\" (UID: \"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15\") "
Sep 17 23:14:30.937954 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:30.937931    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-bundle" (OuterVolumeSpecName: "bundle") pod "7b15067d-6b73-4fb4-a0cb-b1561c1fbd15" (UID: "7b15067d-6b73-4fb4-a0cb-b1561c1fbd15"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:14:30.939026 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:30.939003    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-kube-api-access-g27qs" (OuterVolumeSpecName: "kube-api-access-g27qs") pod "7b15067d-6b73-4fb4-a0cb-b1561c1fbd15" (UID: "7b15067d-6b73-4fb4-a0cb-b1561c1fbd15"). InnerVolumeSpecName "kube-api-access-g27qs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:14:30.942754 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:30.942726    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-util" (OuterVolumeSpecName: "util") pod "7b15067d-6b73-4fb4-a0cb-b1561c1fbd15" (UID: "7b15067d-6b73-4fb4-a0cb-b1561c1fbd15"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:14:31.038536 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:31.038461    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-util\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:14:31.038536 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:31.038494    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g27qs\" (UniqueName: \"kubernetes.io/projected/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-kube-api-access-g27qs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:14:31.038536 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:31.038505    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b15067d-6b73-4fb4-a0cb-b1561c1fbd15-bundle\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:14:31.687815 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:31.687782    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq" event={"ID":"253192e6-6970-47fe-bba9-ab238e390a33","Type":"ContainerStarted","Data":"61346ac606b30bd7ee2dd6d14c974061d2fab95ea6eb09bbaaf88fa0bb552b5e"}
Sep 17 23:14:31.689794 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:31.689769    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw" event={"ID":"7b15067d-6b73-4fb4-a0cb-b1561c1fbd15","Type":"ContainerDied","Data":"539a814de12f674e64500e699143bb8853f29a159e755bd164753d3449e3fcf1"}
Sep 17 23:14:31.689794 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:31.689804    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0d2lbw"
Sep 17 23:14:31.689965 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:31.689810    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="539a814de12f674e64500e699143bb8853f29a159e755bd164753d3449e3fcf1"
Sep 17 23:14:32.695581 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:32.695551    2558 generic.go:356] "Generic (PLEG): container finished" podID="253192e6-6970-47fe-bba9-ab238e390a33" containerID="61346ac606b30bd7ee2dd6d14c974061d2fab95ea6eb09bbaaf88fa0bb552b5e" exitCode=0
Sep 17 23:14:32.695943 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:32.695608    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq" event={"ID":"253192e6-6970-47fe-bba9-ab238e390a33","Type":"ContainerDied","Data":"61346ac606b30bd7ee2dd6d14c974061d2fab95ea6eb09bbaaf88fa0bb552b5e"}
Sep 17 23:14:33.701678 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:33.701644    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq" event={"ID":"253192e6-6970-47fe-bba9-ab238e390a33","Type":"ContainerStarted","Data":"fc09589156607da4bf2663d59067a469ebc6c8ec9bb2b7a766423f68c59c57ab"}
Sep 17 23:14:33.702047 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:33.701745    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:33.724286 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:33.724242    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq" podStartSLOduration=2.327539474 podStartE2EDuration="6.72422841s" totalImagesPullingTime="3.365450508s" totalInitContainerRuntime="1.031238428s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:14:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:14:33.722067908 +0000 UTC m=+497.902706356" watchObservedRunningTime="2026-09-17 23:14:33.72422841 +0000 UTC m=+497.904866847"
Sep 17 23:14:38.904669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:38.904639    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:39.005335 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:39.005299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:41.672348 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:41.672312    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:41.750146 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:41.750103    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:44.708095 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:44.708070    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-747fdf4bf5-t85kq"
Sep 17 23:14:45.150391 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.150324    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"]
Sep 17 23:14:45.150587 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.150575    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b15067d-6b73-4fb4-a0cb-b1561c1fbd15" containerName="extract"
Sep 17 23:14:45.150680 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.150589    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b15067d-6b73-4fb4-a0cb-b1561c1fbd15" containerName="extract"
Sep 17 23:14:45.150680 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.150602    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b15067d-6b73-4fb4-a0cb-b1561c1fbd15" containerName="util"
Sep 17 23:14:45.150680 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.150609    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b15067d-6b73-4fb4-a0cb-b1561c1fbd15" containerName="util"
Sep 17 23:14:45.150680 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.150616    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b15067d-6b73-4fb4-a0cb-b1561c1fbd15" containerName="pull"
Sep 17 23:14:45.150680 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.150627    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b15067d-6b73-4fb4-a0cb-b1561c1fbd15" containerName="pull"
Sep 17 23:14:45.150680 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.150673    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b15067d-6b73-4fb4-a0cb-b1561c1fbd15" containerName="extract"
Sep 17 23:14:45.158902 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.158883    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.161685 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.161662    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"]
Sep 17 23:14:45.162385 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.162362    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 17 23:14:45.162385 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.162380    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 17 23:14:45.162589 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.162387    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 17 23:14:45.162589 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.162412    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 17 23:14:45.162589 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.162424    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 17 23:14:45.162589 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.162372    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bmztk\""
Sep 17 23:14:45.238040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.238020    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5e158797-1120-4201-9b85-4e14d94295fc-manager-config\") pod \"lws-controller-manager-678fd6f85-z6b5t\" (UID: \"5e158797-1120-4201-9b85-4e14d94295fc\") " pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.238129 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.238056    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5e158797-1120-4201-9b85-4e14d94295fc-metrics-cert\") pod \"lws-controller-manager-678fd6f85-z6b5t\" (UID: \"5e158797-1120-4201-9b85-4e14d94295fc\") " pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.238129 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.238073    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw5kj\" (UniqueName: \"kubernetes.io/projected/5e158797-1120-4201-9b85-4e14d94295fc-kube-api-access-dw5kj\") pod \"lws-controller-manager-678fd6f85-z6b5t\" (UID: \"5e158797-1120-4201-9b85-4e14d94295fc\") " pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.238218 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.238157    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e158797-1120-4201-9b85-4e14d94295fc-cert\") pod \"lws-controller-manager-678fd6f85-z6b5t\" (UID: \"5e158797-1120-4201-9b85-4e14d94295fc\") " pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.339048 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.339029    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5e158797-1120-4201-9b85-4e14d94295fc-metrics-cert\") pod \"lws-controller-manager-678fd6f85-z6b5t\" (UID: \"5e158797-1120-4201-9b85-4e14d94295fc\") " pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.339150 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.339056    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw5kj\" (UniqueName: \"kubernetes.io/projected/5e158797-1120-4201-9b85-4e14d94295fc-kube-api-access-dw5kj\") pod \"lws-controller-manager-678fd6f85-z6b5t\" (UID: \"5e158797-1120-4201-9b85-4e14d94295fc\") " pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.339150 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.339088    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e158797-1120-4201-9b85-4e14d94295fc-cert\") pod \"lws-controller-manager-678fd6f85-z6b5t\" (UID: \"5e158797-1120-4201-9b85-4e14d94295fc\") " pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.339268 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.339208    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5e158797-1120-4201-9b85-4e14d94295fc-manager-config\") pod \"lws-controller-manager-678fd6f85-z6b5t\" (UID: \"5e158797-1120-4201-9b85-4e14d94295fc\") " pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.339805 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.339787    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5e158797-1120-4201-9b85-4e14d94295fc-manager-config\") pod \"lws-controller-manager-678fd6f85-z6b5t\" (UID: \"5e158797-1120-4201-9b85-4e14d94295fc\") " pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.341324 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.341302    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e158797-1120-4201-9b85-4e14d94295fc-cert\") pod \"lws-controller-manager-678fd6f85-z6b5t\" (UID: \"5e158797-1120-4201-9b85-4e14d94295fc\") " pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.341450 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.341432    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5e158797-1120-4201-9b85-4e14d94295fc-metrics-cert\") pod \"lws-controller-manager-678fd6f85-z6b5t\" (UID: \"5e158797-1120-4201-9b85-4e14d94295fc\") " pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.358801 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.358777    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw5kj\" (UniqueName: \"kubernetes.io/projected/5e158797-1120-4201-9b85-4e14d94295fc-kube-api-access-dw5kj\") pod \"lws-controller-manager-678fd6f85-z6b5t\" (UID: \"5e158797-1120-4201-9b85-4e14d94295fc\") " pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.468505 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.468484    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:45.744126 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.744031    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"]
Sep 17 23:14:45.746359 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:14:45.746327    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e158797_1120_4201_9b85_4e14d94295fc.slice/crio-8fca6dba9bc0a3f6689f9f8f438d0e836a7e22c138e68c14a26ac8ee7be834d6 WatchSource:0}: Error finding container 8fca6dba9bc0a3f6689f9f8f438d0e836a7e22c138e68c14a26ac8ee7be834d6: Status 404 returned error can't find the container with id 8fca6dba9bc0a3f6689f9f8f438d0e836a7e22c138e68c14a26ac8ee7be834d6
Sep 17 23:14:45.752463 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:45.752417    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t" event={"ID":"5e158797-1120-4201-9b85-4e14d94295fc","Type":"ContainerStarted","Data":"8fca6dba9bc0a3f6689f9f8f438d0e836a7e22c138e68c14a26ac8ee7be834d6"}
Sep 17 23:14:46.265669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:46.265641    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:46.427758 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:46.427715    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:46.682321 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:46.682246    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:47.764279 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:47.764242    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t" event={"ID":"5e158797-1120-4201-9b85-4e14d94295fc","Type":"ContainerStarted","Data":"ef99f5bc46450f909f59182e47e06ca6257fd3106b4af8e781c7433532044a94"}
Sep 17 23:14:47.764780 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:47.764372    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:47.786757 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:47.786714    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t" podStartSLOduration=1.346524081 podStartE2EDuration="2.786701122s" totalImagesPullingTime="1.440177041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:14:47.784618974 +0000 UTC m=+511.965257414" watchObservedRunningTime="2026-09-17 23:14:47.786701122 +0000 UTC m=+511.967339560"
Sep 17 23:14:47.819811 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:47.819767    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:48.113397 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:48.113321    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:48.177105 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:48.177078    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:49.095605 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:49.095574    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:53.091171 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.091135    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:53.199476 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.199452    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:53.647048 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.647014    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"]
Sep 17 23:14:53.651586 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.651567    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.654058 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.654024    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 17 23:14:53.654221 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.654079    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 17 23:14:53.654221 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.654086    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qpjx7\""
Sep 17 23:14:53.654346 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.654236    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 17 23:14:53.660838 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.660814    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"]
Sep 17 23:14:53.674566 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.674522    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:53.684972 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.684949    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.685077 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.684985    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17d06310-4705-4d83-a87d-c63d35283c59-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.685077 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.685005    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.685077 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.685027    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17d06310-4705-4d83-a87d-c63d35283c59-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.685178 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.685113    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47nqg\" (UniqueName: \"kubernetes.io/projected/17d06310-4705-4d83-a87d-c63d35283c59-kube-api-access-47nqg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.685178 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.685157    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.685244 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.685187    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17d06310-4705-4d83-a87d-c63d35283c59-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.685244 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.685218    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.685306 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.685282    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.786153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.786126    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17d06310-4705-4d83-a87d-c63d35283c59-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.786281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.786160    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.786281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.786186    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.786281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.786209    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.786281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.786247    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17d06310-4705-4d83-a87d-c63d35283c59-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.786281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.786274    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.786497 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.786299    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17d06310-4705-4d83-a87d-c63d35283c59-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.786497 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.786344    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47nqg\" (UniqueName: \"kubernetes.io/projected/17d06310-4705-4d83-a87d-c63d35283c59-kube-api-access-47nqg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.786497 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.786367    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.786854 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.786831    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.786950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.786923    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.787074 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.786983    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.787074 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.786992    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.787282 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.787263    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17d06310-4705-4d83-a87d-c63d35283c59-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.788749 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.788722    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17d06310-4705-4d83-a87d-c63d35283c59-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.788965 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.788946    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17d06310-4705-4d83-a87d-c63d35283c59-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.793539 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.793498    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17d06310-4705-4d83-a87d-c63d35283c59-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.794015 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.793995    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47nqg\" (UniqueName: \"kubernetes.io/projected/17d06310-4705-4d83-a87d-c63d35283c59-kube-api-access-47nqg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx\" (UID: \"17d06310-4705-4d83-a87d-c63d35283c59\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:53.867528 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.867494    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:53.963400 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:53.963377    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:54.233801 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:54.233780    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"]
Sep 17 23:14:54.235528 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:14:54.235492    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17d06310_4705_4d83_a87d_c63d35283c59.slice/crio-ce4d28ddaaa47ebffe30bc3652dc730be7cb3cf47b51b2bf081872c4bcde456a WatchSource:0}: Error finding container ce4d28ddaaa47ebffe30bc3652dc730be7cb3cf47b51b2bf081872c4bcde456a: Status 404 returned error can't find the container with id ce4d28ddaaa47ebffe30bc3652dc730be7cb3cf47b51b2bf081872c4bcde456a
Sep 17 23:14:54.255937 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:54.255917    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:54.796692 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:54.796663    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx" event={"ID":"17d06310-4705-4d83-a87d-c63d35283c59","Type":"ContainerStarted","Data":"ce4d28ddaaa47ebffe30bc3652dc730be7cb3cf47b51b2bf081872c4bcde456a"}
Sep 17 23:14:56.729484 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:56.729436    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:14:56.729778 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:56.729528    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:14:56.729778 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:56.729558    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:14:56.807209 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:56.807180    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx" event={"ID":"17d06310-4705-4d83-a87d-c63d35283c59","Type":"ContainerStarted","Data":"886ffaa5171aa51bf569ef74cbc67bc182f12e3a8e436949f4ad58913783c9b1"}
Sep 17 23:14:56.830073 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:56.830028    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx" podStartSLOduration=1.338280798 podStartE2EDuration="3.830012626s" totalImagesPullingTime="2.491731828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:14:56.828991866 +0000 UTC m=+521.009630304" watchObservedRunningTime="2026-09-17 23:14:56.830012626 +0000 UTC m=+521.010651063"
Sep 17 23:14:56.964171 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:56.964147    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:57.320882 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.320855    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r9ddr"]
Sep 17 23:14:57.324059 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.324043    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-r9ddr"
Sep 17 23:14:57.326455 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.326435    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 17 23:14:57.326603 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.326585    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-k27s5\""
Sep 17 23:14:57.335305 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.335283    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r9ddr"]
Sep 17 23:14:57.354856 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.354828    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:57.413971 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.413949    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85bee3c1-36a2-486a-bb72-b48cef7565c9-cert\") pod \"odh-model-controller-7998c67dc4-r9ddr\" (UID: \"85bee3c1-36a2-486a-bb72-b48cef7565c9\") " pod="opendatahub/odh-model-controller-7998c67dc4-r9ddr"
Sep 17 23:14:57.414082 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.413975    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:57.414082 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.413996    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vccmq\" (UniqueName: \"kubernetes.io/projected/85bee3c1-36a2-486a-bb72-b48cef7565c9-kube-api-access-vccmq\") pod \"odh-model-controller-7998c67dc4-r9ddr\" (UID: \"85bee3c1-36a2-486a-bb72-b48cef7565c9\") " pod="opendatahub/odh-model-controller-7998c67dc4-r9ddr"
Sep 17 23:14:57.514084 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.514051    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:57.514339 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.514323    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85bee3c1-36a2-486a-bb72-b48cef7565c9-cert\") pod \"odh-model-controller-7998c67dc4-r9ddr\" (UID: \"85bee3c1-36a2-486a-bb72-b48cef7565c9\") " pod="opendatahub/odh-model-controller-7998c67dc4-r9ddr"
Sep 17 23:14:57.514379 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.514359    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vccmq\" (UniqueName: \"kubernetes.io/projected/85bee3c1-36a2-486a-bb72-b48cef7565c9-kube-api-access-vccmq\") pod \"odh-model-controller-7998c67dc4-r9ddr\" (UID: \"85bee3c1-36a2-486a-bb72-b48cef7565c9\") " pod="opendatahub/odh-model-controller-7998c67dc4-r9ddr"
Sep 17 23:14:57.516654 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.516631    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85bee3c1-36a2-486a-bb72-b48cef7565c9-cert\") pod \"odh-model-controller-7998c67dc4-r9ddr\" (UID: \"85bee3c1-36a2-486a-bb72-b48cef7565c9\") " pod="opendatahub/odh-model-controller-7998c67dc4-r9ddr"
Sep 17 23:14:57.524244 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.524226    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vccmq\" (UniqueName: \"kubernetes.io/projected/85bee3c1-36a2-486a-bb72-b48cef7565c9-kube-api-access-vccmq\") pod \"odh-model-controller-7998c67dc4-r9ddr\" (UID: \"85bee3c1-36a2-486a-bb72-b48cef7565c9\") " pod="opendatahub/odh-model-controller-7998c67dc4-r9ddr"
Sep 17 23:14:57.614429 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.614374    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:57.634427 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.634404    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-r9ddr"
Sep 17 23:14:57.881347 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.881271    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:57.905431 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.905406    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r9ddr"]
Sep 17 23:14:57.908563 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:14:57.908509    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85bee3c1_36a2_486a_bb72_b48cef7565c9.slice/crio-ef518dc24674a4f1a58a838e73a7f589a9a20e996e42bac8cb143d7f625a2a47 WatchSource:0}: Error finding container ef518dc24674a4f1a58a838e73a7f589a9a20e996e42bac8cb143d7f625a2a47: Status 404 returned error can't find the container with id ef518dc24674a4f1a58a838e73a7f589a9a20e996e42bac8cb143d7f625a2a47
Sep 17 23:14:57.968554 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.968506    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:57.980964 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:57.980940    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:14:58.771832 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:58.771797    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-678fd6f85-z6b5t"
Sep 17 23:14:58.820426 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:58.819638    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-r9ddr" event={"ID":"85bee3c1-36a2-486a-bb72-b48cef7565c9","Type":"ContainerStarted","Data":"ef518dc24674a4f1a58a838e73a7f589a9a20e996e42bac8cb143d7f625a2a47"}
Sep 17 23:14:58.820426 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:58.820378    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:14:58.821365 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:14:58.821172    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx"
Sep 17 23:15:00.831182 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:00.831151    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-r9ddr" event={"ID":"85bee3c1-36a2-486a-bb72-b48cef7565c9","Type":"ContainerStarted","Data":"e4584188eb6bba988378b81253b3f2bd289b7735365e804e0869a0f79d16e2e3"}
Sep 17 23:15:00.831557 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:00.831301    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-r9ddr"
Sep 17 23:15:00.846914 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:00.846819    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-r9ddr" podStartSLOduration=1.168807521 podStartE2EDuration="3.846801475s" totalImagesPullingTime="2.677993954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:14:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:15:00.84661203 +0000 UTC m=+525.027250470" watchObservedRunningTime="2026-09-17 23:15:00.846801475 +0000 UTC m=+525.027439914"
Sep 17 23:15:03.990796 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:03.990764    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"]
Sep 17 23:15:03.993728 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:03.993712    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:03.997151 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:03.997130    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 17 23:15:03.997649 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:03.997632    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6t4sq\""
Sep 17 23:15:03.997731 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:03.997713    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 17 23:15:03.997803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:03.997761    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 17 23:15:04.020320 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.020298    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"]
Sep 17 23:15:04.067699 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.067668    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wndmq\" (UniqueName: \"kubernetes.io/projected/f7ff21c7-43e7-4085-855d-03e61bd04eb5-kube-api-access-wndmq\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.067699 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.067698    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f7ff21c7-43e7-4085-855d-03e61bd04eb5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.067847 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.067723    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7ff21c7-43e7-4085-855d-03e61bd04eb5-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.067847 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.067746    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7ff21c7-43e7-4085-855d-03e61bd04eb5-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.067847 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.067796    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f7ff21c7-43e7-4085-855d-03e61bd04eb5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.067957 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.067848    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7ff21c7-43e7-4085-855d-03e61bd04eb5-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.067957 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.067874    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f7ff21c7-43e7-4085-855d-03e61bd04eb5-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.168324 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.168298    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7ff21c7-43e7-4085-855d-03e61bd04eb5-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.168475 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.168336    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7ff21c7-43e7-4085-855d-03e61bd04eb5-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.168475 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.168358    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f7ff21c7-43e7-4085-855d-03e61bd04eb5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.168475 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.168394    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7ff21c7-43e7-4085-855d-03e61bd04eb5-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.168475 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.168423    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f7ff21c7-43e7-4085-855d-03e61bd04eb5-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.168475 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.168470    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wndmq\" (UniqueName: \"kubernetes.io/projected/f7ff21c7-43e7-4085-855d-03e61bd04eb5-kube-api-access-wndmq\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.168776 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.168494    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f7ff21c7-43e7-4085-855d-03e61bd04eb5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.169288 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.169262    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f7ff21c7-43e7-4085-855d-03e61bd04eb5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.170914 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.170888    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7ff21c7-43e7-4085-855d-03e61bd04eb5-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.170990 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.170935    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f7ff21c7-43e7-4085-855d-03e61bd04eb5-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.171034 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.171024    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7ff21c7-43e7-4085-855d-03e61bd04eb5-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.171119 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.171100    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f7ff21c7-43e7-4085-855d-03e61bd04eb5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.180817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.180792    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wndmq\" (UniqueName: \"kubernetes.io/projected/f7ff21c7-43e7-4085-855d-03e61bd04eb5-kube-api-access-wndmq\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.181651 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.181070    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7ff21c7-43e7-4085-855d-03e61bd04eb5-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cz26c\" (UID: \"f7ff21c7-43e7-4085-855d-03e61bd04eb5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.302567 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.302467    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:04.577832 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.577754    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"]
Sep 17 23:15:04.580571 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:15:04.580500    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7ff21c7_43e7_4085_855d_03e61bd04eb5.slice/crio-99167630672ead91126b68f2a40fddd2d3c6141f9a518c9e3f759f390bc53d52 WatchSource:0}: Error finding container 99167630672ead91126b68f2a40fddd2d3c6141f9a518c9e3f759f390bc53d52: Status 404 returned error can't find the container with id 99167630672ead91126b68f2a40fddd2d3c6141f9a518c9e3f759f390bc53d52
Sep 17 23:15:04.851864 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:04.851795    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c" event={"ID":"f7ff21c7-43e7-4085-855d-03e61bd04eb5","Type":"ContainerStarted","Data":"99167630672ead91126b68f2a40fddd2d3c6141f9a518c9e3f759f390bc53d52"}
Sep 17 23:15:06.448947 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:06.448912    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:06.611397 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:06.611336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:06.815993 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:06.815918    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:06.912920 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:06.912873    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:07.242437 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:07.242402    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:15:07.242506 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:07.242482    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:15:07.782180 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:07.782146    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:07.853983 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:07.853954    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:07.866533 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:07.866483    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c" event={"ID":"f7ff21c7-43e7-4085-855d-03e61bd04eb5","Type":"ContainerStarted","Data":"000cf1011256711677e3afeced5e78f789c56644ac9ddbc4dbf4e1d0bfeab59e"}
Sep 17 23:15:07.866665 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:07.866563    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:07.890744 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:07.890702    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c" podStartSLOduration=2.231031667 podStartE2EDuration="4.890689184s" totalImagesPullingTime="2.659657517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:15:07.888836711 +0000 UTC m=+532.069475169" watchObservedRunningTime="2026-09-17 23:15:07.890689184 +0000 UTC m=+532.071327623"
Sep 17 23:15:08.635498 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:08.635468    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:15:08.872777 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:08.872753    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cz26c"
Sep 17 23:15:11.846011 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:11.845977    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-r9ddr"
Sep 17 23:15:58.347293 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:15:58.347263    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:00.042290 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:00.042250    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:01.973736 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:01.973704    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"]
Sep 17 23:16:01.976848 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:01.976831    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"
Sep 17 23:16:01.979190 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:01.979165    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-86r94\""
Sep 17 23:16:01.979325 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:01.979211    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 23:16:01.979941 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:01.979927    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 23:16:01.985769 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:01.985740    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"]
Sep 17 23:16:02.065907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.065868    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnhw5\" (UniqueName: \"kubernetes.io/projected/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-kube-api-access-bnhw5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6\" (UID: \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"
Sep 17 23:16:02.066090 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.065939    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6\" (UID: \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"
Sep 17 23:16:02.066090 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.065988    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6\" (UID: \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"
Sep 17 23:16:02.068339 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.068316    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"]
Sep 17 23:16:02.071642 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.071627    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"
Sep 17 23:16:02.080950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.080918    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"]
Sep 17 23:16:02.167011 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.166977    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6\" (UID: \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"
Sep 17 23:16:02.167011 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.167017    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d549f38-eebe-432b-a7d9-a37b0dff9251-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489\" (UID: \"1d549f38-eebe-432b-a7d9-a37b0dff9251\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"
Sep 17 23:16:02.167225 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.167036    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v98xp\" (UniqueName: \"kubernetes.io/projected/1d549f38-eebe-432b-a7d9-a37b0dff9251-kube-api-access-v98xp\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489\" (UID: \"1d549f38-eebe-432b-a7d9-a37b0dff9251\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"
Sep 17 23:16:02.167225 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.167059    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6\" (UID: \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"
Sep 17 23:16:02.167225 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.167143    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnhw5\" (UniqueName: \"kubernetes.io/projected/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-kube-api-access-bnhw5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6\" (UID: \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"
Sep 17 23:16:02.167225 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.167173    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d549f38-eebe-432b-a7d9-a37b0dff9251-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489\" (UID: \"1d549f38-eebe-432b-a7d9-a37b0dff9251\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"
Sep 17 23:16:02.167425 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.167408    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6\" (UID: \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"
Sep 17 23:16:02.167461 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.167447    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6\" (UID: \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"
Sep 17 23:16:02.170367 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.170344    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"]
Sep 17 23:16:02.173668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.173654    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"
Sep 17 23:16:02.178001 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.177981    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnhw5\" (UniqueName: \"kubernetes.io/projected/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-kube-api-access-bnhw5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6\" (UID: \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"
Sep 17 23:16:02.182581 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.182563    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"]
Sep 17 23:16:02.267767 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.267690    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c84p4\" (UniqueName: \"kubernetes.io/projected/38a79b52-263c-4a5c-a3c3-6287bafd642c-kube-api-access-c84p4\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49\" (UID: \"38a79b52-263c-4a5c-a3c3-6287bafd642c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"
Sep 17 23:16:02.267767 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.267734    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38a79b52-263c-4a5c-a3c3-6287bafd642c-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49\" (UID: \"38a79b52-263c-4a5c-a3c3-6287bafd642c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"
Sep 17 23:16:02.267767 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.267762    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38a79b52-263c-4a5c-a3c3-6287bafd642c-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49\" (UID: \"38a79b52-263c-4a5c-a3c3-6287bafd642c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"
Sep 17 23:16:02.267975 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.267842    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d549f38-eebe-432b-a7d9-a37b0dff9251-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489\" (UID: \"1d549f38-eebe-432b-a7d9-a37b0dff9251\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"
Sep 17 23:16:02.267975 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.267920    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d549f38-eebe-432b-a7d9-a37b0dff9251-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489\" (UID: \"1d549f38-eebe-432b-a7d9-a37b0dff9251\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"
Sep 17 23:16:02.267975 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.267949    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v98xp\" (UniqueName: \"kubernetes.io/projected/1d549f38-eebe-432b-a7d9-a37b0dff9251-kube-api-access-v98xp\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489\" (UID: \"1d549f38-eebe-432b-a7d9-a37b0dff9251\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"
Sep 17 23:16:02.268234 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.268219    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d549f38-eebe-432b-a7d9-a37b0dff9251-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489\" (UID: \"1d549f38-eebe-432b-a7d9-a37b0dff9251\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"
Sep 17 23:16:02.268293 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.268259    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d549f38-eebe-432b-a7d9-a37b0dff9251-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489\" (UID: \"1d549f38-eebe-432b-a7d9-a37b0dff9251\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"
Sep 17 23:16:02.269339 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.269319    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"]
Sep 17 23:16:02.272944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.272930    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"
Sep 17 23:16:02.277463 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.277437    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v98xp\" (UniqueName: \"kubernetes.io/projected/1d549f38-eebe-432b-a7d9-a37b0dff9251-kube-api-access-v98xp\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489\" (UID: \"1d549f38-eebe-432b-a7d9-a37b0dff9251\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"
Sep 17 23:16:02.280537 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.280493    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"]
Sep 17 23:16:02.286739 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.286715    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"
Sep 17 23:16:02.368683 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.368654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38a79b52-263c-4a5c-a3c3-6287bafd642c-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49\" (UID: \"38a79b52-263c-4a5c-a3c3-6287bafd642c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"
Sep 17 23:16:02.368683 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.368684    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38a79b52-263c-4a5c-a3c3-6287bafd642c-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49\" (UID: \"38a79b52-263c-4a5c-a3c3-6287bafd642c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"
Sep 17 23:16:02.368857 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.368754    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6d7r\" (UniqueName: \"kubernetes.io/projected/5c78820f-ffe0-4060-be99-a60fdfac2652-kube-api-access-w6d7r\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq\" (UID: \"5c78820f-ffe0-4060-be99-a60fdfac2652\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"
Sep 17 23:16:02.368857 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.368780    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c78820f-ffe0-4060-be99-a60fdfac2652-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq\" (UID: \"5c78820f-ffe0-4060-be99-a60fdfac2652\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"
Sep 17 23:16:02.368857 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.368812    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c78820f-ffe0-4060-be99-a60fdfac2652-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq\" (UID: \"5c78820f-ffe0-4060-be99-a60fdfac2652\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"
Sep 17 23:16:02.368857 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.368843    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c84p4\" (UniqueName: \"kubernetes.io/projected/38a79b52-263c-4a5c-a3c3-6287bafd642c-kube-api-access-c84p4\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49\" (UID: \"38a79b52-263c-4a5c-a3c3-6287bafd642c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"
Sep 17 23:16:02.369074 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.369049    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38a79b52-263c-4a5c-a3c3-6287bafd642c-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49\" (UID: \"38a79b52-263c-4a5c-a3c3-6287bafd642c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"
Sep 17 23:16:02.369157 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.369143    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38a79b52-263c-4a5c-a3c3-6287bafd642c-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49\" (UID: \"38a79b52-263c-4a5c-a3c3-6287bafd642c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"
Sep 17 23:16:02.376712 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.376685    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c84p4\" (UniqueName: \"kubernetes.io/projected/38a79b52-263c-4a5c-a3c3-6287bafd642c-kube-api-access-c84p4\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49\" (UID: \"38a79b52-263c-4a5c-a3c3-6287bafd642c\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"
Sep 17 23:16:02.381486 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.381470    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"
Sep 17 23:16:02.469768 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.469738    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c78820f-ffe0-4060-be99-a60fdfac2652-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq\" (UID: \"5c78820f-ffe0-4060-be99-a60fdfac2652\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"
Sep 17 23:16:02.469966 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.469778    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c78820f-ffe0-4060-be99-a60fdfac2652-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq\" (UID: \"5c78820f-ffe0-4060-be99-a60fdfac2652\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"
Sep 17 23:16:02.469966 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.469854    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6d7r\" (UniqueName: \"kubernetes.io/projected/5c78820f-ffe0-4060-be99-a60fdfac2652-kube-api-access-w6d7r\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq\" (UID: \"5c78820f-ffe0-4060-be99-a60fdfac2652\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"
Sep 17 23:16:02.470194 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.470171    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c78820f-ffe0-4060-be99-a60fdfac2652-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq\" (UID: \"5c78820f-ffe0-4060-be99-a60fdfac2652\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"
Sep 17 23:16:02.470259 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.470188    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c78820f-ffe0-4060-be99-a60fdfac2652-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq\" (UID: \"5c78820f-ffe0-4060-be99-a60fdfac2652\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"
Sep 17 23:16:02.478262 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.478241    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6d7r\" (UniqueName: \"kubernetes.io/projected/5c78820f-ffe0-4060-be99-a60fdfac2652-kube-api-access-w6d7r\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq\" (UID: \"5c78820f-ffe0-4060-be99-a60fdfac2652\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"
Sep 17 23:16:02.490102 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.490081    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"
Sep 17 23:16:02.563827 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.563796    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"]
Sep 17 23:16:02.566680 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:16:02.566654    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09dd20aa_24e0_42ab_9ca7_65cdc5dde697.slice/crio-b7af03c3f5b1da8b5fd912a5e1c6ee4a06c5390a94da903456cdfb4eb097614c WatchSource:0}: Error finding container b7af03c3f5b1da8b5fd912a5e1c6ee4a06c5390a94da903456cdfb4eb097614c: Status 404 returned error can't find the container with id b7af03c3f5b1da8b5fd912a5e1c6ee4a06c5390a94da903456cdfb4eb097614c
Sep 17 23:16:02.589660 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.589639    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"
Sep 17 23:16:02.656391 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.656345    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"]
Sep 17 23:16:02.656661 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:16:02.656596    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d549f38_eebe_432b_a7d9_a37b0dff9251.slice/crio-a65b9d1d864f7951f55f36d86b1ff7c0c43972459293f6126bb0c0ab201b7da1 WatchSource:0}: Error finding container a65b9d1d864f7951f55f36d86b1ff7c0c43972459293f6126bb0c0ab201b7da1: Status 404 returned error can't find the container with id a65b9d1d864f7951f55f36d86b1ff7c0c43972459293f6126bb0c0ab201b7da1
Sep 17 23:16:02.759722 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.759696    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"]
Sep 17 23:16:02.760902 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:16:02.760879    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a79b52_263c_4a5c_a3c3_6287bafd642c.slice/crio-4b37e709de212aeae8f618c5393c27477ad92b2bc050fa0c52d72177ef79c655 WatchSource:0}: Error finding container 4b37e709de212aeae8f618c5393c27477ad92b2bc050fa0c52d72177ef79c655: Status 404 returned error can't find the container with id 4b37e709de212aeae8f618c5393c27477ad92b2bc050fa0c52d72177ef79c655
Sep 17 23:16:02.861733 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:02.861709    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"]
Sep 17 23:16:02.907254 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:16:02.907231    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c78820f_ffe0_4060_be99_a60fdfac2652.slice/crio-f809ae848a48b6f1d3be59a57af36cd4b0ecfc5ca772b26725db373e37a84b5a WatchSource:0}: Error finding container f809ae848a48b6f1d3be59a57af36cd4b0ecfc5ca772b26725db373e37a84b5a: Status 404 returned error can't find the container with id f809ae848a48b6f1d3be59a57af36cd4b0ecfc5ca772b26725db373e37a84b5a
Sep 17 23:16:03.103134 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:03.103081    2558 generic.go:356] "Generic (PLEG): container finished" podID="38a79b52-263c-4a5c-a3c3-6287bafd642c" containerID="19d105877f30e6f229dc9b6835c8e60b2858214f90fe6dece65056c294fdb30a" exitCode=0
Sep 17 23:16:03.103465 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:03.103168    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49" event={"ID":"38a79b52-263c-4a5c-a3c3-6287bafd642c","Type":"ContainerDied","Data":"19d105877f30e6f229dc9b6835c8e60b2858214f90fe6dece65056c294fdb30a"}
Sep 17 23:16:03.103465 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:03.103224    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49" event={"ID":"38a79b52-263c-4a5c-a3c3-6287bafd642c","Type":"ContainerStarted","Data":"4b37e709de212aeae8f618c5393c27477ad92b2bc050fa0c52d72177ef79c655"}
Sep 17 23:16:03.104790 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:03.104769    2558 generic.go:356] "Generic (PLEG): container finished" podID="1d549f38-eebe-432b-a7d9-a37b0dff9251" containerID="5629856f5f36a8c20a87a545e9d453a431d28314aa4c68e487dec7450a9b5cf8" exitCode=0
Sep 17 23:16:03.104889 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:03.104853    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489" event={"ID":"1d549f38-eebe-432b-a7d9-a37b0dff9251","Type":"ContainerDied","Data":"5629856f5f36a8c20a87a545e9d453a431d28314aa4c68e487dec7450a9b5cf8"}
Sep 17 23:16:03.104953 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:03.104901    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489" event={"ID":"1d549f38-eebe-432b-a7d9-a37b0dff9251","Type":"ContainerStarted","Data":"a65b9d1d864f7951f55f36d86b1ff7c0c43972459293f6126bb0c0ab201b7da1"}
Sep 17 23:16:03.106453 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:03.106431    2558 generic.go:356] "Generic (PLEG): container finished" podID="09dd20aa-24e0-42ab-9ca7-65cdc5dde697" containerID="319aa6327272fd4361dc9f2766112ec42bbcfd7e84d250bbd6b6c1ee872414d9" exitCode=0
Sep 17 23:16:03.106556 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:03.106533    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6" event={"ID":"09dd20aa-24e0-42ab-9ca7-65cdc5dde697","Type":"ContainerDied","Data":"319aa6327272fd4361dc9f2766112ec42bbcfd7e84d250bbd6b6c1ee872414d9"}
Sep 17 23:16:03.106623 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:03.106572    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6" event={"ID":"09dd20aa-24e0-42ab-9ca7-65cdc5dde697","Type":"ContainerStarted","Data":"b7af03c3f5b1da8b5fd912a5e1c6ee4a06c5390a94da903456cdfb4eb097614c"}
Sep 17 23:16:03.108420 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:03.108399    2558 generic.go:356] "Generic (PLEG): container finished" podID="5c78820f-ffe0-4060-be99-a60fdfac2652" containerID="5cd93ba36fd0beb921a36d219322d3722644cf93eae16ef380d4cffb4ed2ef5c" exitCode=0
Sep 17 23:16:03.108523 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:03.108481    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq" event={"ID":"5c78820f-ffe0-4060-be99-a60fdfac2652","Type":"ContainerDied","Data":"5cd93ba36fd0beb921a36d219322d3722644cf93eae16ef380d4cffb4ed2ef5c"}
Sep 17 23:16:03.108592 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:03.108540    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq" event={"ID":"5c78820f-ffe0-4060-be99-a60fdfac2652","Type":"ContainerStarted","Data":"f809ae848a48b6f1d3be59a57af36cd4b0ecfc5ca772b26725db373e37a84b5a"}
Sep 17 23:16:04.117628 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:04.117598    2558 generic.go:356] "Generic (PLEG): container finished" podID="1d549f38-eebe-432b-a7d9-a37b0dff9251" containerID="52801d066351a02a66dedc6862c57a47cedc9103688729c58aecdcb47cc51bcc" exitCode=0
Sep 17 23:16:04.117959 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:04.117672    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489" event={"ID":"1d549f38-eebe-432b-a7d9-a37b0dff9251","Type":"ContainerDied","Data":"52801d066351a02a66dedc6862c57a47cedc9103688729c58aecdcb47cc51bcc"}
Sep 17 23:16:04.392693 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:16:04.392655    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c78820f_ffe0_4060_be99_a60fdfac2652.slice/crio-conmon-d5f485964ec67ad72116bfebe0348098908519955f87578ece7ed79a4261508c.scope\": RecentStats: unable to find data in memory cache]"
Sep 17 23:16:04.392693 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:16:04.392683    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c78820f_ffe0_4060_be99_a60fdfac2652.slice/crio-conmon-d5f485964ec67ad72116bfebe0348098908519955f87578ece7ed79a4261508c.scope\": RecentStats: unable to find data in memory cache]"
Sep 17 23:16:05.124885 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:05.124848    2558 generic.go:356] "Generic (PLEG): container finished" podID="1d549f38-eebe-432b-a7d9-a37b0dff9251" containerID="cc0f84c96887fe80b3ec25b501b476833fefc516d9f3088a95f2ea72723ff98c" exitCode=0
Sep 17 23:16:05.125295 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:05.124934    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489" event={"ID":"1d549f38-eebe-432b-a7d9-a37b0dff9251","Type":"ContainerDied","Data":"cc0f84c96887fe80b3ec25b501b476833fefc516d9f3088a95f2ea72723ff98c"}
Sep 17 23:16:05.126586 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:05.126564    2558 generic.go:356] "Generic (PLEG): container finished" podID="09dd20aa-24e0-42ab-9ca7-65cdc5dde697" containerID="e0566cc13708633a6a34d495610cf0e3d47098a3623101ac20ef129e98902abd" exitCode=0
Sep 17 23:16:05.126706 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:05.126652    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6" event={"ID":"09dd20aa-24e0-42ab-9ca7-65cdc5dde697","Type":"ContainerDied","Data":"e0566cc13708633a6a34d495610cf0e3d47098a3623101ac20ef129e98902abd"}
Sep 17 23:16:05.128473 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:05.128453    2558 generic.go:356] "Generic (PLEG): container finished" podID="5c78820f-ffe0-4060-be99-a60fdfac2652" containerID="d5f485964ec67ad72116bfebe0348098908519955f87578ece7ed79a4261508c" exitCode=0
Sep 17 23:16:05.128574 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:05.128534    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq" event={"ID":"5c78820f-ffe0-4060-be99-a60fdfac2652","Type":"ContainerDied","Data":"d5f485964ec67ad72116bfebe0348098908519955f87578ece7ed79a4261508c"}
Sep 17 23:16:05.130572 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:05.130553    2558 generic.go:356] "Generic (PLEG): container finished" podID="38a79b52-263c-4a5c-a3c3-6287bafd642c" containerID="6656d7682aa16d562aba3e186472f7ba322f53edac87ed09eb82ff80c4b48dcd" exitCode=0
Sep 17 23:16:05.130675 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:05.130589    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49" event={"ID":"38a79b52-263c-4a5c-a3c3-6287bafd642c","Type":"ContainerDied","Data":"6656d7682aa16d562aba3e186472f7ba322f53edac87ed09eb82ff80c4b48dcd"}
Sep 17 23:16:06.137612 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.137576    2558 generic.go:356] "Generic (PLEG): container finished" podID="09dd20aa-24e0-42ab-9ca7-65cdc5dde697" containerID="697911e97f4199706a995645c704fc9dbf33189f4f1d10398e4f751be64b4a10" exitCode=0
Sep 17 23:16:06.137985 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.137667    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6" event={"ID":"09dd20aa-24e0-42ab-9ca7-65cdc5dde697","Type":"ContainerDied","Data":"697911e97f4199706a995645c704fc9dbf33189f4f1d10398e4f751be64b4a10"}
Sep 17 23:16:06.139610 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.139588    2558 generic.go:356] "Generic (PLEG): container finished" podID="5c78820f-ffe0-4060-be99-a60fdfac2652" containerID="1a8e7ef1b7d5cf651d813e18d3c1419220adcf31b8149d354d8e837d7b362e58" exitCode=0
Sep 17 23:16:06.139739 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.139665    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq" event={"ID":"5c78820f-ffe0-4060-be99-a60fdfac2652","Type":"ContainerDied","Data":"1a8e7ef1b7d5cf651d813e18d3c1419220adcf31b8149d354d8e837d7b362e58"}
Sep 17 23:16:06.141717 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.141696    2558 generic.go:356] "Generic (PLEG): container finished" podID="38a79b52-263c-4a5c-a3c3-6287bafd642c" containerID="9ec0e5e937fdcccedcb4ba0741cf44a35a049cb7eb7024e16c8f35106cbaf410" exitCode=0
Sep 17 23:16:06.141800 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.141767    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49" event={"ID":"38a79b52-263c-4a5c-a3c3-6287bafd642c","Type":"ContainerDied","Data":"9ec0e5e937fdcccedcb4ba0741cf44a35a049cb7eb7024e16c8f35106cbaf410"}
Sep 17 23:16:06.275342 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.275318    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"
Sep 17 23:16:06.403286 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.403210    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v98xp\" (UniqueName: \"kubernetes.io/projected/1d549f38-eebe-432b-a7d9-a37b0dff9251-kube-api-access-v98xp\") pod \"1d549f38-eebe-432b-a7d9-a37b0dff9251\" (UID: \"1d549f38-eebe-432b-a7d9-a37b0dff9251\") "
Sep 17 23:16:06.403286 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.403265    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d549f38-eebe-432b-a7d9-a37b0dff9251-util\") pod \"1d549f38-eebe-432b-a7d9-a37b0dff9251\" (UID: \"1d549f38-eebe-432b-a7d9-a37b0dff9251\") "
Sep 17 23:16:06.403476 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.403295    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d549f38-eebe-432b-a7d9-a37b0dff9251-bundle\") pod \"1d549f38-eebe-432b-a7d9-a37b0dff9251\" (UID: \"1d549f38-eebe-432b-a7d9-a37b0dff9251\") "
Sep 17 23:16:06.403944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.403919    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d549f38-eebe-432b-a7d9-a37b0dff9251-bundle" (OuterVolumeSpecName: "bundle") pod "1d549f38-eebe-432b-a7d9-a37b0dff9251" (UID: "1d549f38-eebe-432b-a7d9-a37b0dff9251"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:16:06.405205 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.405169    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d549f38-eebe-432b-a7d9-a37b0dff9251-kube-api-access-v98xp" (OuterVolumeSpecName: "kube-api-access-v98xp") pod "1d549f38-eebe-432b-a7d9-a37b0dff9251" (UID: "1d549f38-eebe-432b-a7d9-a37b0dff9251"). InnerVolumeSpecName "kube-api-access-v98xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:16:06.408636 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.408599    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d549f38-eebe-432b-a7d9-a37b0dff9251-util" (OuterVolumeSpecName: "util") pod "1d549f38-eebe-432b-a7d9-a37b0dff9251" (UID: "1d549f38-eebe-432b-a7d9-a37b0dff9251"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:16:06.504679 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.504646    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d549f38-eebe-432b-a7d9-a37b0dff9251-util\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:16:06.504679 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.504678    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d549f38-eebe-432b-a7d9-a37b0dff9251-bundle\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:16:06.504805 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:06.504689    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v98xp\" (UniqueName: \"kubernetes.io/projected/1d549f38-eebe-432b-a7d9-a37b0dff9251-kube-api-access-v98xp\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:16:07.148483 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.148458    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489"
Sep 17 23:16:07.148854 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.148504    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agf489" event={"ID":"1d549f38-eebe-432b-a7d9-a37b0dff9251","Type":"ContainerDied","Data":"a65b9d1d864f7951f55f36d86b1ff7c0c43972459293f6126bb0c0ab201b7da1"}
Sep 17 23:16:07.148854 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.148560    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a65b9d1d864f7951f55f36d86b1ff7c0c43972459293f6126bb0c0ab201b7da1"
Sep 17 23:16:07.292331 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.292308    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"
Sep 17 23:16:07.327345 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.327323    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"
Sep 17 23:16:07.330270 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.330251    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"
Sep 17 23:16:07.409285 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.409230    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnhw5\" (UniqueName: \"kubernetes.io/projected/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-kube-api-access-bnhw5\") pod \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\" (UID: \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\") "
Sep 17 23:16:07.409285 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.409269    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c84p4\" (UniqueName: \"kubernetes.io/projected/38a79b52-263c-4a5c-a3c3-6287bafd642c-kube-api-access-c84p4\") pod \"38a79b52-263c-4a5c-a3c3-6287bafd642c\" (UID: \"38a79b52-263c-4a5c-a3c3-6287bafd642c\") "
Sep 17 23:16:07.409467 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.409296    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38a79b52-263c-4a5c-a3c3-6287bafd642c-util\") pod \"38a79b52-263c-4a5c-a3c3-6287bafd642c\" (UID: \"38a79b52-263c-4a5c-a3c3-6287bafd642c\") "
Sep 17 23:16:07.409467 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.409415    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c78820f-ffe0-4060-be99-a60fdfac2652-util\") pod \"5c78820f-ffe0-4060-be99-a60fdfac2652\" (UID: \"5c78820f-ffe0-4060-be99-a60fdfac2652\") "
Sep 17 23:16:07.409467 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.409462    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-util\") pod \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\" (UID: \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\") "
Sep 17 23:16:07.409656 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.409486    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c78820f-ffe0-4060-be99-a60fdfac2652-bundle\") pod \"5c78820f-ffe0-4060-be99-a60fdfac2652\" (UID: \"5c78820f-ffe0-4060-be99-a60fdfac2652\") "
Sep 17 23:16:07.409656 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.409567    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-bundle\") pod \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\" (UID: \"09dd20aa-24e0-42ab-9ca7-65cdc5dde697\") "
Sep 17 23:16:07.409656 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.409606    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6d7r\" (UniqueName: \"kubernetes.io/projected/5c78820f-ffe0-4060-be99-a60fdfac2652-kube-api-access-w6d7r\") pod \"5c78820f-ffe0-4060-be99-a60fdfac2652\" (UID: \"5c78820f-ffe0-4060-be99-a60fdfac2652\") "
Sep 17 23:16:07.409656 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.409641    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38a79b52-263c-4a5c-a3c3-6287bafd642c-bundle\") pod \"38a79b52-263c-4a5c-a3c3-6287bafd642c\" (UID: \"38a79b52-263c-4a5c-a3c3-6287bafd642c\") "
Sep 17 23:16:07.410395 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.409999    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-bundle" (OuterVolumeSpecName: "bundle") pod "09dd20aa-24e0-42ab-9ca7-65cdc5dde697" (UID: "09dd20aa-24e0-42ab-9ca7-65cdc5dde697"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:16:07.410395 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.410020    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c78820f-ffe0-4060-be99-a60fdfac2652-bundle" (OuterVolumeSpecName: "bundle") pod "5c78820f-ffe0-4060-be99-a60fdfac2652" (UID: "5c78820f-ffe0-4060-be99-a60fdfac2652"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:16:07.410580 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.410418    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38a79b52-263c-4a5c-a3c3-6287bafd642c-bundle" (OuterVolumeSpecName: "bundle") pod "38a79b52-263c-4a5c-a3c3-6287bafd642c" (UID: "38a79b52-263c-4a5c-a3c3-6287bafd642c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:16:07.411866 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.411840    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38a79b52-263c-4a5c-a3c3-6287bafd642c-kube-api-access-c84p4" (OuterVolumeSpecName: "kube-api-access-c84p4") pod "38a79b52-263c-4a5c-a3c3-6287bafd642c" (UID: "38a79b52-263c-4a5c-a3c3-6287bafd642c"). InnerVolumeSpecName "kube-api-access-c84p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:16:07.412181 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.412153    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-kube-api-access-bnhw5" (OuterVolumeSpecName: "kube-api-access-bnhw5") pod "09dd20aa-24e0-42ab-9ca7-65cdc5dde697" (UID: "09dd20aa-24e0-42ab-9ca7-65cdc5dde697"). InnerVolumeSpecName "kube-api-access-bnhw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:16:07.412741 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.412720    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c78820f-ffe0-4060-be99-a60fdfac2652-kube-api-access-w6d7r" (OuterVolumeSpecName: "kube-api-access-w6d7r") pod "5c78820f-ffe0-4060-be99-a60fdfac2652" (UID: "5c78820f-ffe0-4060-be99-a60fdfac2652"). InnerVolumeSpecName "kube-api-access-w6d7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:16:07.415452 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.415422    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38a79b52-263c-4a5c-a3c3-6287bafd642c-util" (OuterVolumeSpecName: "util") pod "38a79b52-263c-4a5c-a3c3-6287bafd642c" (UID: "38a79b52-263c-4a5c-a3c3-6287bafd642c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:16:07.415576 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.415558    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c78820f-ffe0-4060-be99-a60fdfac2652-util" (OuterVolumeSpecName: "util") pod "5c78820f-ffe0-4060-be99-a60fdfac2652" (UID: "5c78820f-ffe0-4060-be99-a60fdfac2652"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:16:07.418482 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.418433    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-util" (OuterVolumeSpecName: "util") pod "09dd20aa-24e0-42ab-9ca7-65cdc5dde697" (UID: "09dd20aa-24e0-42ab-9ca7-65cdc5dde697"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:16:07.510803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.510776    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c84p4\" (UniqueName: \"kubernetes.io/projected/38a79b52-263c-4a5c-a3c3-6287bafd642c-kube-api-access-c84p4\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:16:07.510803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.510800    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/38a79b52-263c-4a5c-a3c3-6287bafd642c-util\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:16:07.510950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.510816    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c78820f-ffe0-4060-be99-a60fdfac2652-util\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:16:07.510950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.510827    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-util\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:16:07.510950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.510838    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c78820f-ffe0-4060-be99-a60fdfac2652-bundle\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:16:07.510950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.510849    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-bundle\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:16:07.510950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.510860    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6d7r\" (UniqueName: \"kubernetes.io/projected/5c78820f-ffe0-4060-be99-a60fdfac2652-kube-api-access-w6d7r\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:16:07.510950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.510872    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/38a79b52-263c-4a5c-a3c3-6287bafd642c-bundle\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:16:07.510950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:07.510885    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnhw5\" (UniqueName: \"kubernetes.io/projected/09dd20aa-24e0-42ab-9ca7-65cdc5dde697-kube-api-access-bnhw5\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:16:08.155712 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:08.155690    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq"
Sep 17 23:16:08.156082 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:08.155717    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jtwkq" event={"ID":"5c78820f-ffe0-4060-be99-a60fdfac2652","Type":"ContainerDied","Data":"f809ae848a48b6f1d3be59a57af36cd4b0ecfc5ca772b26725db373e37a84b5a"}
Sep 17 23:16:08.156082 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:08.155768    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f809ae848a48b6f1d3be59a57af36cd4b0ecfc5ca772b26725db373e37a84b5a"
Sep 17 23:16:08.158142 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:08.158118    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49" event={"ID":"38a79b52-263c-4a5c-a3c3-6287bafd642c","Type":"ContainerDied","Data":"4b37e709de212aeae8f618c5393c27477ad92b2bc050fa0c52d72177ef79c655"}
Sep 17 23:16:08.158271 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:08.158164    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b37e709de212aeae8f618c5393c27477ad92b2bc050fa0c52d72177ef79c655"
Sep 17 23:16:08.158271 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:08.158203    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnbg49"
Sep 17 23:16:08.160989 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:08.160957    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6" event={"ID":"09dd20aa-24e0-42ab-9ca7-65cdc5dde697","Type":"ContainerDied","Data":"b7af03c3f5b1da8b5fd912a5e1c6ee4a06c5390a94da903456cdfb4eb097614c"}
Sep 17 23:16:08.161170 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:08.161154    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7af03c3f5b1da8b5fd912a5e1c6ee4a06c5390a94da903456cdfb4eb097614c"
Sep 17 23:16:08.161710 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:08.161687    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c626cn6"
Sep 17 23:16:15.163250 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:15.163220    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:17.315489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:17.315450    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:17.373944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:17.373918    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:17.465204 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:17.465177    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:17.522451 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:17.522429    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:18.094959 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:18.094915    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:18.216698 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:18.216668    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:18.761882 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:18.761776    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:19.360931 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:19.360902    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:19.971066 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:19.971036    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:24.354104 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:24.354066    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:24.967346 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:24.967315    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:27.967946 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:27.967916    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:28.070636 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:28.070604    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:28.167826 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:28.167798    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:28.267904 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:28.267838    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:28.638560 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:28.638470    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:31.209689 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:31.209619    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:31.364184 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:31.364151    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:32.473761 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:32.473733    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:32.574328 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:32.574297    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:33.007345 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:33.007315    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:34.197540 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:34.197489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:34.422569 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:34.422532    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:34.522210 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:34.522145    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:35.015946 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:35.015915    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:35.634847 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:35.634794    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:35.741871 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:35.741843    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:36.367753 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:36.367723    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:36.429255 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:36.429232    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:36.775887 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:36.775858    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:36.832825 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:36.832795    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:37.167928 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:37.167854    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:37.268347 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:37.268319    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:45.603422 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:45.603390    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:45.700488 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:45.700459    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:48.029710 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:48.029672    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:48.084204 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:48.084158    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:48.477300 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:48.477274    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:52.720109 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:52.720077    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:52.820917 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:52.820889    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:55.713325 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:55.713289    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:55.811128 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:55.811101    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:16:55.935536 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:16:55.935497    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:04.536348 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:04.536316    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:04.637165 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:04.637134    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:07.010871 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:07.010840    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:07.107168 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:07.107139    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:10.349334 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:10.349300    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:10.869362 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:10.869326    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:11.349115 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:11.349082    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:11.449627 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:11.449599    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:12.655923 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:12.655890    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:15.156323 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:15.156290    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:15.257189 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:15.257162    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:15.355738 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:15.355709    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:15.416836 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:15.416773    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:16.342896 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:16.342869    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:16.442274 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:16.442244    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:25.563155 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:25.563121    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:25.663316 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:25.663287    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:26.663879 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:26.663845    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:27.626191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:27.626151    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:27.729947 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:27.729889    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:27.831285 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:27.831251    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:28.003701 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:28.003668    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:28.104126 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:28.104094    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:28.465926 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:28.465890    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:28.566295 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:28.566266    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:45.429013 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:45.428968    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:45.524675 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:45.524639    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:45.910016 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:45.909988    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:58.596431 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:58.596392    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:17:58.758146 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:17:58.758111    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:05.106556 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.106458    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:05.134017 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.133990    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b8d44d488-5kp9h"]
Sep 17 23:18:05.134308 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134297    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d549f38-eebe-432b-a7d9-a37b0dff9251" containerName="pull"
Sep 17 23:18:05.134351 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134310    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d549f38-eebe-432b-a7d9-a37b0dff9251" containerName="pull"
Sep 17 23:18:05.134351 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134318    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c78820f-ffe0-4060-be99-a60fdfac2652" containerName="pull"
Sep 17 23:18:05.134351 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134323    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c78820f-ffe0-4060-be99-a60fdfac2652" containerName="pull"
Sep 17 23:18:05.134351 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134332    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c78820f-ffe0-4060-be99-a60fdfac2652" containerName="extract"
Sep 17 23:18:05.134351 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134338    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c78820f-ffe0-4060-be99-a60fdfac2652" containerName="extract"
Sep 17 23:18:05.134351 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134344    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c78820f-ffe0-4060-be99-a60fdfac2652" containerName="util"
Sep 17 23:18:05.134351 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134349    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c78820f-ffe0-4060-be99-a60fdfac2652" containerName="util"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134354    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d549f38-eebe-432b-a7d9-a37b0dff9251" containerName="extract"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134359    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d549f38-eebe-432b-a7d9-a37b0dff9251" containerName="extract"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134364    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09dd20aa-24e0-42ab-9ca7-65cdc5dde697" containerName="extract"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134369    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="09dd20aa-24e0-42ab-9ca7-65cdc5dde697" containerName="extract"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134378    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09dd20aa-24e0-42ab-9ca7-65cdc5dde697" containerName="util"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134382    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="09dd20aa-24e0-42ab-9ca7-65cdc5dde697" containerName="util"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134387    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09dd20aa-24e0-42ab-9ca7-65cdc5dde697" containerName="pull"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134392    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="09dd20aa-24e0-42ab-9ca7-65cdc5dde697" containerName="pull"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134400    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38a79b52-263c-4a5c-a3c3-6287bafd642c" containerName="util"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134405    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="38a79b52-263c-4a5c-a3c3-6287bafd642c" containerName="util"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134411    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38a79b52-263c-4a5c-a3c3-6287bafd642c" containerName="pull"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134416    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="38a79b52-263c-4a5c-a3c3-6287bafd642c" containerName="pull"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134423    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38a79b52-263c-4a5c-a3c3-6287bafd642c" containerName="extract"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134428    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="38a79b52-263c-4a5c-a3c3-6287bafd642c" containerName="extract"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134436    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d549f38-eebe-432b-a7d9-a37b0dff9251" containerName="util"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134440    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d549f38-eebe-432b-a7d9-a37b0dff9251" containerName="util"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134489    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="09dd20aa-24e0-42ab-9ca7-65cdc5dde697" containerName="extract"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134497    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="38a79b52-263c-4a5c-a3c3-6287bafd642c" containerName="extract"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134503    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d549f38-eebe-432b-a7d9-a37b0dff9251" containerName="extract"
Sep 17 23:18:05.134585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.134508    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c78820f-ffe0-4060-be99-a60fdfac2652" containerName="extract"
Sep 17 23:18:05.137550 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.137508    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:05.140144 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.140110    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 17 23:18:05.140272 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.140189    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2mntz\""
Sep 17 23:18:05.140443 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.140428    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 17 23:18:05.141027 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.141004    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 17 23:18:05.147783 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.147731    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b8d44d488-5kp9h"]
Sep 17 23:18:05.173547 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.173501    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:05.189681 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.189652    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-metrics-certs\") pod \"maas-controller-5b8d44d488-5kp9h\" (UID: \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\") " pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:05.189815 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.189715    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxm5s\" (UniqueName: \"kubernetes.io/projected/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-kube-api-access-sxm5s\") pod \"maas-controller-5b8d44d488-5kp9h\" (UID: \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\") " pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:05.189815 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.189757    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-webhook-cert\") pod \"maas-controller-5b8d44d488-5kp9h\" (UID: \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\") " pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:05.230310 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.230285    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:05.291103 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.291070    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxm5s\" (UniqueName: \"kubernetes.io/projected/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-kube-api-access-sxm5s\") pod \"maas-controller-5b8d44d488-5kp9h\" (UID: \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\") " pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:05.291290 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.291141    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-webhook-cert\") pod \"maas-controller-5b8d44d488-5kp9h\" (UID: \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\") " pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:05.291290 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.291193    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-metrics-certs\") pod \"maas-controller-5b8d44d488-5kp9h\" (UID: \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\") " pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:05.291415 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:18:05.291300    2558 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 17 23:18:05.291415 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:18:05.291389    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-webhook-cert podName:c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe nodeName:}" failed. No retries permitted until 2026-09-17 23:18:05.791367103 +0000 UTC m=+709.972005539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-webhook-cert") pod "maas-controller-5b8d44d488-5kp9h" (UID: "c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe") : secret "maas-controller-webhook-cert" not found
Sep 17 23:18:05.293725 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.293704    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-metrics-certs\") pod \"maas-controller-5b8d44d488-5kp9h\" (UID: \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\") " pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:05.295983 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.295960    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:05.304406 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.304384    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxm5s\" (UniqueName: \"kubernetes.io/projected/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-kube-api-access-sxm5s\") pod \"maas-controller-5b8d44d488-5kp9h\" (UID: \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\") " pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:05.554096 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.554061    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:05.794908 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.794871    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-webhook-cert\") pod \"maas-controller-5b8d44d488-5kp9h\" (UID: \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\") " pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:05.797233 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:05.797202    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-webhook-cert\") pod \"maas-controller-5b8d44d488-5kp9h\" (UID: \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\") " pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:06.049557 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:06.049510    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:06.318662 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:06.318638    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b8d44d488-5kp9h"]
Sep 17 23:18:06.320452 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:18:06.320418    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc84f8ece_648d_4db3_a5f7_c91a8d0c5dfe.slice/crio-d69f20276d134ad3bcdc73a7a1ccae2ca48d8fc2885a1b780309dddcafcd9d5b WatchSource:0}: Error finding container d69f20276d134ad3bcdc73a7a1ccae2ca48d8fc2885a1b780309dddcafcd9d5b: Status 404 returned error can't find the container with id d69f20276d134ad3bcdc73a7a1ccae2ca48d8fc2885a1b780309dddcafcd9d5b
Sep 17 23:18:06.732335 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:06.732302    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b8d44d488-5kp9h" event={"ID":"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe","Type":"ContainerStarted","Data":"d69f20276d134ad3bcdc73a7a1ccae2ca48d8fc2885a1b780309dddcafcd9d5b"}
Sep 17 23:18:08.745061 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:08.745032    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b8d44d488-5kp9h" event={"ID":"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe","Type":"ContainerStarted","Data":"6c16f43310c26a60bfff407e3ecda04cb94cee4023298afb74c47f53c7fe3535"}
Sep 17 23:18:08.745453 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:08.745142    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:08.763943 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:08.763889    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b8d44d488-5kp9h" podStartSLOduration=1.458118012 podStartE2EDuration="3.76387204s" totalImagesPullingTime="2.305754028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:18:08.76153523 +0000 UTC m=+712.942173667" watchObservedRunningTime="2026-09-17 23:18:08.76387204 +0000 UTC m=+712.944510480"
Sep 17 23:18:08.829614 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:08.829576    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:08.931622 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:08.931538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:17.389999 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:17.389965    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:17.450245 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:17.450213    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:19.752433 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:19.752397    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:20.409484 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.409450    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b8d44d488-5kp9h"]
Sep 17 23:18:20.409689 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.409662    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5b8d44d488-5kp9h" podUID="c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe" containerName="manager" containerID="cri-o://6c16f43310c26a60bfff407e3ecda04cb94cee4023298afb74c47f53c7fe3535" gracePeriod=30
Sep 17 23:18:20.647984 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.647961    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:20.724766 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.724738    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxm5s\" (UniqueName: \"kubernetes.io/projected/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-kube-api-access-sxm5s\") pod \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\" (UID: \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\") "
Sep 17 23:18:20.724919 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.724809    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-metrics-certs\") pod \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\" (UID: \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\") "
Sep 17 23:18:20.728733 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.726482    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-webhook-cert\") pod \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\" (UID: \"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe\") "
Sep 17 23:18:20.728733 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.728055    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-kube-api-access-sxm5s" (OuterVolumeSpecName: "kube-api-access-sxm5s") pod "c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe" (UID: "c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe"). InnerVolumeSpecName "kube-api-access-sxm5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:18:20.732197 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.732172    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe" (UID: "c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:18:20.732197 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.732184    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe" (UID: "c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:18:20.808455 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.808419    2558 generic.go:356] "Generic (PLEG): container finished" podID="c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe" containerID="6c16f43310c26a60bfff407e3ecda04cb94cee4023298afb74c47f53c7fe3535" exitCode=0
Sep 17 23:18:20.808949 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.808481    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b8d44d488-5kp9h"
Sep 17 23:18:20.808949 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.808509    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b8d44d488-5kp9h" event={"ID":"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe","Type":"ContainerDied","Data":"6c16f43310c26a60bfff407e3ecda04cb94cee4023298afb74c47f53c7fe3535"}
Sep 17 23:18:20.808949 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.808618    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b8d44d488-5kp9h" event={"ID":"c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe","Type":"ContainerDied","Data":"d69f20276d134ad3bcdc73a7a1ccae2ca48d8fc2885a1b780309dddcafcd9d5b"}
Sep 17 23:18:20.808949 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.808651    2558 scope.go:117] "RemoveContainer" containerID="6c16f43310c26a60bfff407e3ecda04cb94cee4023298afb74c47f53c7fe3535"
Sep 17 23:18:20.817392 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.817375    2558 scope.go:117] "RemoveContainer" containerID="6c16f43310c26a60bfff407e3ecda04cb94cee4023298afb74c47f53c7fe3535"
Sep 17 23:18:20.817698 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:18:20.817680    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c16f43310c26a60bfff407e3ecda04cb94cee4023298afb74c47f53c7fe3535\": container with ID starting with 6c16f43310c26a60bfff407e3ecda04cb94cee4023298afb74c47f53c7fe3535 not found: ID does not exist" containerID="6c16f43310c26a60bfff407e3ecda04cb94cee4023298afb74c47f53c7fe3535"
Sep 17 23:18:20.817747 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.817706    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c16f43310c26a60bfff407e3ecda04cb94cee4023298afb74c47f53c7fe3535"} err="failed to get container status \"6c16f43310c26a60bfff407e3ecda04cb94cee4023298afb74c47f53c7fe3535\": rpc error: code = NotFound desc = could not find container \"6c16f43310c26a60bfff407e3ecda04cb94cee4023298afb74c47f53c7fe3535\": container with ID starting with 6c16f43310c26a60bfff407e3ecda04cb94cee4023298afb74c47f53c7fe3535 not found: ID does not exist"
Sep 17 23:18:20.827563 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.827473    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxm5s\" (UniqueName: \"kubernetes.io/projected/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-kube-api-access-sxm5s\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:18:20.827563 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.827500    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:18:20.827563 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.827538    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe-webhook-cert\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:18:20.829782 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.829763    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b8d44d488-5kp9h"]
Sep 17 23:18:20.835608 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:20.835583    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b8d44d488-5kp9h"]
Sep 17 23:18:21.931835 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:21.931802    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe" path="/var/lib/kubelet/pods/c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe/volumes"
Sep 17 23:18:29.460705 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:29.460672    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:40.390843 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:40.389310    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:40.489216 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:40.489181    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:41.847843 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:41.847798    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:41.949796 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:41.949767    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:42.048461 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.048425    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:42.083039 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.083005    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"]
Sep 17 23:18:42.083440 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.083426    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe" containerName="manager"
Sep 17 23:18:42.083486 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.083443    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe" containerName="manager"
Sep 17 23:18:42.083545 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.083534    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c84f8ece-648d-4db3-a5f7-c91a8d0c5dfe" containerName="manager"
Sep 17 23:18:42.090042 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.090022    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.092602 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.092582    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 17 23:18:42.093551 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.093527    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 17 23:18:42.093875 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.093855    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 17 23:18:42.094008 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.093880    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-skqc5\""
Sep 17 23:18:42.094073 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.093894    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 23:18:42.096701 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.096680    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"]
Sep 17 23:18:42.112284 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.112219    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thlhg\" (UniqueName: \"kubernetes.io/projected/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-kube-api-access-thlhg\") pod \"maas-api-6678c8db5f-2l4wl\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") " pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.112284 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.112261    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-openshift-service-ca\") pod \"maas-api-6678c8db5f-2l4wl\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") " pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.112466 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.112297    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-maas-api-tls\") pod \"maas-api-6678c8db5f-2l4wl\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") " pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.112466 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.112387    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-metrics-certs\") pod \"maas-api-6678c8db5f-2l4wl\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") " pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.149185 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.149161    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:42.213441 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.213411    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-maas-api-tls\") pod \"maas-api-6678c8db5f-2l4wl\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") " pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.213629 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.213461    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-metrics-certs\") pod \"maas-api-6678c8db5f-2l4wl\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") " pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.213629 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.213552    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thlhg\" (UniqueName: \"kubernetes.io/projected/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-kube-api-access-thlhg\") pod \"maas-api-6678c8db5f-2l4wl\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") " pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.213629 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.213579    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-openshift-service-ca\") pod \"maas-api-6678c8db5f-2l4wl\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") " pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.214229 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.214194    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-openshift-service-ca\") pod \"maas-api-6678c8db5f-2l4wl\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") " pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.215907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.215886    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-metrics-certs\") pod \"maas-api-6678c8db5f-2l4wl\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") " pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.216051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.215884    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-maas-api-tls\") pod \"maas-api-6678c8db5f-2l4wl\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") " pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.221123 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.221100    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thlhg\" (UniqueName: \"kubernetes.io/projected/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-kube-api-access-thlhg\") pod \"maas-api-6678c8db5f-2l4wl\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") " pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.359287 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.359251    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:42.402021 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.401952    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:42.458591 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.458558    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:42.681125 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.681072    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"]
Sep 17 23:18:42.683470 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:18:42.683443    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7d6c6f6_3464_4212_8a1d_08cdb3d889ef.slice/crio-a5f406fefa534e0e297a103c7217da129141479d583f44c2966967de86653a7d WatchSource:0}: Error finding container a5f406fefa534e0e297a103c7217da129141479d583f44c2966967de86653a7d: Status 404 returned error can't find the container with id a5f406fefa534e0e297a103c7217da129141479d583f44c2966967de86653a7d
Sep 17 23:18:42.684873 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.684855    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:18:42.912182 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:42.912143    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl" event={"ID":"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef","Type":"ContainerStarted","Data":"a5f406fefa534e0e297a103c7217da129141479d583f44c2966967de86653a7d"}
Sep 17 23:18:44.925840 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:44.925806    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl" event={"ID":"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef","Type":"ContainerStarted","Data":"369bfabd73d0c03c7e93a5c9af006d8bf378c4882f546a831505e60b2fbd0194"}
Sep 17 23:18:44.926284 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:44.925881    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:44.941616 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:44.941570    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl" podStartSLOduration=1.2522292560000001 podStartE2EDuration="2.94155538s" totalImagesPullingTime="1.689326124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:18:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:18:44.940177784 +0000 UTC m=+749.120816222" watchObservedRunningTime="2026-09-17 23:18:44.94155538 +0000 UTC m=+749.122193871"
Sep 17 23:18:50.936397 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:50.936370    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:18:52.184292 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:52.184249    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:18:52.283918 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:18:52.283880    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:03.639752 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.639720    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"]
Sep 17 23:19:03.642825 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.642802    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.645736 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.645716    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 17 23:19:03.646626 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.646592    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jks5l\""
Sep 17 23:19:03.646733 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.646636    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 17 23:19:03.646733 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.646600    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 17 23:19:03.652489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.652465    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"]
Sep 17 23:19:03.664348 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.664328    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:03.794769 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.794729    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bebec92-9bb5-42b1-b380-3b52a133ce6e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.794769 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.794774    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bebec92-9bb5-42b1-b380-3b52a133ce6e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.794973 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.794798    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bebec92-9bb5-42b1-b380-3b52a133ce6e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.794973 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.794815    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kljtv\" (UniqueName: \"kubernetes.io/projected/6bebec92-9bb5-42b1-b380-3b52a133ce6e-kube-api-access-kljtv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.794973 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.794861    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bebec92-9bb5-42b1-b380-3b52a133ce6e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.794973 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.794877    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bebec92-9bb5-42b1-b380-3b52a133ce6e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.895870 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.895790    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bebec92-9bb5-42b1-b380-3b52a133ce6e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.895870 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.895825    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bebec92-9bb5-42b1-b380-3b52a133ce6e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.895870 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.895851    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bebec92-9bb5-42b1-b380-3b52a133ce6e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.896100 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.895874    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kljtv\" (UniqueName: \"kubernetes.io/projected/6bebec92-9bb5-42b1-b380-3b52a133ce6e-kube-api-access-kljtv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.896100 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.895923    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bebec92-9bb5-42b1-b380-3b52a133ce6e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.896100 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.895948    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bebec92-9bb5-42b1-b380-3b52a133ce6e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.896233 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.896208    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bebec92-9bb5-42b1-b380-3b52a133ce6e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.896323 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.896300    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bebec92-9bb5-42b1-b380-3b52a133ce6e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.896364 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.896328    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bebec92-9bb5-42b1-b380-3b52a133ce6e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.898164 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.898135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bebec92-9bb5-42b1-b380-3b52a133ce6e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.898358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.898343    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bebec92-9bb5-42b1-b380-3b52a133ce6e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.903959 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.903932    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kljtv\" (UniqueName: \"kubernetes.io/projected/6bebec92-9bb5-42b1-b380-3b52a133ce6e-kube-api-access-kljtv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2\" (UID: \"6bebec92-9bb5-42b1-b380-3b52a133ce6e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:03.953659 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:03.953632    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:04.229246 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:04.229214    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"]
Sep 17 23:19:04.230332 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:19:04.230309    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bebec92_9bb5_42b1_b380_3b52a133ce6e.slice/crio-bcc122e2b965ca668247f522579f8a3a6dafafb025294d32ba1c3ef58a86b08e WatchSource:0}: Error finding container bcc122e2b965ca668247f522579f8a3a6dafafb025294d32ba1c3ef58a86b08e: Status 404 returned error can't find the container with id bcc122e2b965ca668247f522579f8a3a6dafafb025294d32ba1c3ef58a86b08e
Sep 17 23:19:05.026707 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:05.026662    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2" event={"ID":"6bebec92-9bb5-42b1-b380-3b52a133ce6e","Type":"ContainerStarted","Data":"bcc122e2b965ca668247f522579f8a3a6dafafb025294d32ba1c3ef58a86b08e"}
Sep 17 23:19:06.032596 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:06.032563    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2" event={"ID":"6bebec92-9bb5-42b1-b380-3b52a133ce6e","Type":"ContainerStarted","Data":"56d1001a763b928618fd24de37a683f5de35178cdcee00304c3afc8a28a1036c"}
Sep 17 23:19:06.049023 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:06.048968    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2" podStartSLOduration=2.079909963 podStartE2EDuration="3.048929455s" totalImagesPullingTime="969.019492ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:19:06.048009203 +0000 UTC m=+770.228647641" watchObservedRunningTime="2026-09-17 23:19:06.048929455 +0000 UTC m=+770.229567892"
Sep 17 23:19:07.070230 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:07.070200    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:13.954134 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:13.954102    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:13.954657 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:13.954166    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:13.966610 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:13.966586    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:14.082865 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:14.082831    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2"
Sep 17 23:19:24.953887 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:24.953853    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:32.623191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.623111    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"]
Sep 17 23:19:32.627847 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.627824    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.630117 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.630098    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 17 23:19:32.635540 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.635498    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"]
Sep 17 23:19:32.656831 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.656805    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:32.739173 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.739143    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhxkp\" (UniqueName: \"kubernetes.io/projected/6d930dce-88ac-4544-b825-71427c0e05c8-kube-api-access-hhxkp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.739316 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.739192    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d930dce-88ac-4544-b825-71427c0e05c8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.739316 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.739265    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d930dce-88ac-4544-b825-71427c0e05c8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.739316 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.739291    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d930dce-88ac-4544-b825-71427c0e05c8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.739316 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.739309    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d930dce-88ac-4544-b825-71427c0e05c8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.739472 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.739343    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d930dce-88ac-4544-b825-71427c0e05c8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.839748 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.839712    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d930dce-88ac-4544-b825-71427c0e05c8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.839903 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.839765    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d930dce-88ac-4544-b825-71427c0e05c8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.839903 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.839784    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d930dce-88ac-4544-b825-71427c0e05c8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.839903 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.839802    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d930dce-88ac-4544-b825-71427c0e05c8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.839903 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.839842    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d930dce-88ac-4544-b825-71427c0e05c8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.839903 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.839870    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhxkp\" (UniqueName: \"kubernetes.io/projected/6d930dce-88ac-4544-b825-71427c0e05c8-kube-api-access-hhxkp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.840208 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.840175    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d930dce-88ac-4544-b825-71427c0e05c8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.840344 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.840234    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d930dce-88ac-4544-b825-71427c0e05c8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.840344 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.840264    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d930dce-88ac-4544-b825-71427c0e05c8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.842108 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.842087    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d930dce-88ac-4544-b825-71427c0e05c8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.842234 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.842216    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d930dce-88ac-4544-b825-71427c0e05c8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.848269 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.848247    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhxkp\" (UniqueName: \"kubernetes.io/projected/6d930dce-88ac-4544-b825-71427c0e05c8-kube-api-access-hhxkp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8sp7c\" (UID: \"6d930dce-88ac-4544-b825-71427c0e05c8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:32.938659 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:32.938637    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:33.209052 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:33.209024    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"]
Sep 17 23:19:33.210466 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:19:33.210441    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d930dce_88ac_4544_b825_71427c0e05c8.slice/crio-b1e5eb91de1ffd58e2fe82898f866eb0f083a91bc2a610777bda97461c9125a3 WatchSource:0}: Error finding container b1e5eb91de1ffd58e2fe82898f866eb0f083a91bc2a610777bda97461c9125a3: Status 404 returned error can't find the container with id b1e5eb91de1ffd58e2fe82898f866eb0f083a91bc2a610777bda97461c9125a3
Sep 17 23:19:34.172764 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:34.172729    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c" event={"ID":"6d930dce-88ac-4544-b825-71427c0e05c8","Type":"ContainerStarted","Data":"ee1192761aa4edde94f1131a7cea5561179d0c19614756a6abf272e385693c22"}
Sep 17 23:19:34.173156 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:34.172785    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c" event={"ID":"6d930dce-88ac-4544-b825-71427c0e05c8","Type":"ContainerStarted","Data":"b1e5eb91de1ffd58e2fe82898f866eb0f083a91bc2a610777bda97461c9125a3"}
Sep 17 23:19:34.192098 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:34.191973    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c" podStartSLOduration=2.028002875 podStartE2EDuration="2.19195625s" totalImagesPullingTime="163.953375ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:19:34.191774702 +0000 UTC m=+798.372413141" watchObservedRunningTime="2026-09-17 23:19:34.19195625 +0000 UTC m=+798.372594689"
Sep 17 23:19:40.429577 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.429540    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"]
Sep 17 23:19:40.432285 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.432261    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.434749 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.434726    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 17 23:19:40.444024 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.444000    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"]
Sep 17 23:19:40.455741 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.455719    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:40.511476 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.511452    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.511611 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.511482    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.511611 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.511501    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.511692 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.511629    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.511692 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.511663    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmczp\" (UniqueName: \"kubernetes.io/projected/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-kube-api-access-gmczp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.511763 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.511732    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.612529 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.612474    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.612683 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.612563    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.612683 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.612582    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.612683 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.612599    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.612683 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.612624    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.612875 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.612732    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmczp\" (UniqueName: \"kubernetes.io/projected/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-kube-api-access-gmczp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.613034 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.613006    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.613154 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.613047    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.613154 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.613083    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.614856 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.614835    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.615106 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.615090    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.620007 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.619985    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmczp\" (UniqueName: \"kubernetes.io/projected/3ccfbb47-c4a8-4ce0-90ad-496e117682cd-kube-api-access-gmczp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5\" (UID: \"3ccfbb47-c4a8-4ce0-90ad-496e117682cd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:40.743487 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:40.743446    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:41.010883 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:41.010857    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"]
Sep 17 23:19:41.012656 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:19:41.012627    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ccfbb47_c4a8_4ce0_90ad_496e117682cd.slice/crio-4c48a3e809132e3c83342d30527cdcdb53e345156e6b994d1cbc11587f7bfa28 WatchSource:0}: Error finding container 4c48a3e809132e3c83342d30527cdcdb53e345156e6b994d1cbc11587f7bfa28: Status 404 returned error can't find the container with id 4c48a3e809132e3c83342d30527cdcdb53e345156e6b994d1cbc11587f7bfa28
Sep 17 23:19:41.208303 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:41.208277    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5" event={"ID":"3ccfbb47-c4a8-4ce0-90ad-496e117682cd","Type":"ContainerStarted","Data":"4c48a3e809132e3c83342d30527cdcdb53e345156e6b994d1cbc11587f7bfa28"}
Sep 17 23:19:42.215345 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:42.215302    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5" event={"ID":"3ccfbb47-c4a8-4ce0-90ad-496e117682cd","Type":"ContainerStarted","Data":"e95cb3243c02815bb1f61c26e9e8b565f587380a9db99202345cad72bd441277"}
Sep 17 23:19:42.232751 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:42.232701    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5" podStartSLOduration=2.05449177 podStartE2EDuration="2.232686638s" totalImagesPullingTime="178.194868ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:19:42.232126349 +0000 UTC m=+806.412764788" watchObservedRunningTime="2026-09-17 23:19:42.232686638 +0000 UTC m=+806.413325115"
Sep 17 23:19:42.939120 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:42.939085    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:42.939380 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:42.939360    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:42.951455 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:42.951433    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:43.231172 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:43.231089    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8sp7c"
Sep 17 23:19:46.727912 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:46.727877    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:50.743816 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:50.743778    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:50.744197 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:50.743841    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:50.755696 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:50.755675    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:51.270793 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:51.270764    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5"
Sep 17 23:19:54.827819 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:54.827782    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:57.713686 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:57.713653    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:19:57.814211 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:19:57.814181    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:16.921193 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:16.921148    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:18.101187 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:18.101152    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:18.197011 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:18.196978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:33.835571 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:33.835502    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:33.940949 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:33.940916    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:34.036035 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:34.035996    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:34.185116 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:34.185041    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:34.272932 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:34.272899    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:34.395527 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:34.395463    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:34.531440 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:34.531410    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:34.853667 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:34.853572    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:34.954007 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:34.953956    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:35.322329 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:35.322286    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:35.418164 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:35.418128    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:35.808800 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:35.808767    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:37.473150 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:37.473107    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:37.562231 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:37.562199    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:37.666630 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:37.666591    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:37.840224 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:37.840131    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:37.960926 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:37.960894    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:38.624445 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.624416    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"]
Sep 17 23:21:38.627934 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.627903    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:38.630817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.630792    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-861522-serving-cert\""
Sep 17 23:21:38.641566 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.641541    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"]
Sep 17 23:21:38.663851 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.663818    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:38.727549 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.727507    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcpvj\" (UniqueName: \"kubernetes.io/projected/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-kube-api-access-hcpvj\") pod \"maas-api-e2e-worker-w0-861522-867898b758-g586b\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:38.727684 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.727575    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-metrics-certs\") pod \"maas-api-e2e-worker-w0-861522-867898b758-g586b\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:38.727684 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.727623    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-maas-api-tls\") pod \"maas-api-e2e-worker-w0-861522-867898b758-g586b\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:38.727684 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.727681    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-861522-867898b758-g586b\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:38.728332 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.728301    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:38.829536 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.829429    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcpvj\" (UniqueName: \"kubernetes.io/projected/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-kube-api-access-hcpvj\") pod \"maas-api-e2e-worker-w0-861522-867898b758-g586b\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:38.830143 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.830119    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-metrics-certs\") pod \"maas-api-e2e-worker-w0-861522-867898b758-g586b\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:38.830228 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.830192    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-maas-api-tls\") pod \"maas-api-e2e-worker-w0-861522-867898b758-g586b\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:38.830304 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.830287    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-861522-867898b758-g586b\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:38.831111 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.831085    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-861522-867898b758-g586b\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:38.831238 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:21:38.831222    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-serving-cert: secret "maas-api-e2e-worker-w0-861522-serving-cert" not found
Sep 17 23:21:38.831297 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:21:38.831291    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-maas-api-tls podName:9a4659d1-bc4f-4fe9-9572-ac673f28a3be nodeName:}" failed. No retries permitted until 2026-09-17 23:21:39.331267765 +0000 UTC m=+923.511906198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-maas-api-tls") pod "maas-api-e2e-worker-w0-861522-867898b758-g586b" (UID: "9a4659d1-bc4f-4fe9-9572-ac673f28a3be") : secret "maas-api-e2e-worker-w0-861522-serving-cert" not found
Sep 17 23:21:38.833212 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.833179    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-metrics-certs\") pod \"maas-api-e2e-worker-w0-861522-867898b758-g586b\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:38.839622 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:38.839598    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcpvj\" (UniqueName: \"kubernetes.io/projected/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-kube-api-access-hcpvj\") pod \"maas-api-e2e-worker-w0-861522-867898b758-g586b\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:39.032474 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.032451    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:39.327408 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.327334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:39.334607 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.334588    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-maas-api-tls\") pod \"maas-api-e2e-worker-w0-861522-867898b758-g586b\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:39.336800 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.336774    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-maas-api-tls\") pod \"maas-api-e2e-worker-w0-861522-867898b758-g586b\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:39.423641 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.423611    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:39.541638 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.541612    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:39.724847 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.724812    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"]
Sep 17 23:21:39.728450 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.728430    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:39.732722 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.732698    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-983490-serving-cert\""
Sep 17 23:21:39.741534 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.741495    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"]
Sep 17 23:21:39.767397 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.767377    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:39.820287 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.820266    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"]
Sep 17 23:21:39.822210 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:21:39.822183    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a4659d1_bc4f_4fe9_9572_ac673f28a3be.slice/crio-8402a80d6a68df78140d49fa9674035ad5b2a4b88921a7688eaa084328071fb9 WatchSource:0}: Error finding container 8402a80d6a68df78140d49fa9674035ad5b2a4b88921a7688eaa084328071fb9: Status 404 returned error can't find the container with id 8402a80d6a68df78140d49fa9674035ad5b2a4b88921a7688eaa084328071fb9
Sep 17 23:21:39.839397 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.839369    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:39.839588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.839554    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb987c4b-f861-46c3-a2de-4c74a918f649-metrics-certs\") pod \"maas-api-e2e-worker-w4-983490-8b856bd79-6x76x\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:39.839719 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.839607    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb987c4b-f861-46c3-a2de-4c74a918f649-maas-api-tls\") pod \"maas-api-e2e-worker-w4-983490-8b856bd79-6x76x\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:39.839719 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.839638    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb987c4b-f861-46c3-a2de-4c74a918f649-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-983490-8b856bd79-6x76x\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:39.839719 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.839688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcdd2\" (UniqueName: \"kubernetes.io/projected/eb987c4b-f861-46c3-a2de-4c74a918f649-kube-api-access-pcdd2\") pod \"maas-api-e2e-worker-w4-983490-8b856bd79-6x76x\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:39.940153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.940124    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb987c4b-f861-46c3-a2de-4c74a918f649-maas-api-tls\") pod \"maas-api-e2e-worker-w4-983490-8b856bd79-6x76x\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:39.940264 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.940181    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb987c4b-f861-46c3-a2de-4c74a918f649-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-983490-8b856bd79-6x76x\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:39.940264 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.940232    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcdd2\" (UniqueName: \"kubernetes.io/projected/eb987c4b-f861-46c3-a2de-4c74a918f649-kube-api-access-pcdd2\") pod \"maas-api-e2e-worker-w4-983490-8b856bd79-6x76x\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:39.940338 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.940258    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:39.940338 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.940313    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb987c4b-f861-46c3-a2de-4c74a918f649-metrics-certs\") pod \"maas-api-e2e-worker-w4-983490-8b856bd79-6x76x\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:39.940987 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.940956    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb987c4b-f861-46c3-a2de-4c74a918f649-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-983490-8b856bd79-6x76x\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:39.942772 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.942751    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb987c4b-f861-46c3-a2de-4c74a918f649-metrics-certs\") pod \"maas-api-e2e-worker-w4-983490-8b856bd79-6x76x\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:39.942882 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.942751    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb987c4b-f861-46c3-a2de-4c74a918f649-maas-api-tls\") pod \"maas-api-e2e-worker-w4-983490-8b856bd79-6x76x\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:39.948045 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:39.948022    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcdd2\" (UniqueName: \"kubernetes.io/projected/eb987c4b-f861-46c3-a2de-4c74a918f649-kube-api-access-pcdd2\") pod \"maas-api-e2e-worker-w4-983490-8b856bd79-6x76x\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:40.039976 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.039953    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:40.321803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.321771    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:40.322172 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.322153    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"]
Sep 17 23:21:40.322499 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:21:40.322481    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb987c4b_f861_46c3_a2de_4c74a918f649.slice/crio-2fcc04fa8f983aec10b5f300306cf5ec86dd84a552882483bd4ddd43544f2c5d WatchSource:0}: Error finding container 2fcc04fa8f983aec10b5f300306cf5ec86dd84a552882483bd4ddd43544f2c5d: Status 404 returned error can't find the container with id 2fcc04fa8f983aec10b5f300306cf5ec86dd84a552882483bd4ddd43544f2c5d
Sep 17 23:21:40.474153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.474082    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:40.799973 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.799886    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x" event={"ID":"eb987c4b-f861-46c3-a2de-4c74a918f649","Type":"ContainerStarted","Data":"9ccad52791b182170be43fc710d36a3ac54c024293ce830403801b515dea81ef"}
Sep 17 23:21:40.799973 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.799949    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x" event={"ID":"eb987c4b-f861-46c3-a2de-4c74a918f649","Type":"ContainerStarted","Data":"2fcc04fa8f983aec10b5f300306cf5ec86dd84a552882483bd4ddd43544f2c5d"}
Sep 17 23:21:40.800385 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.799988    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:40.801504 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.801483    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b" event={"ID":"9a4659d1-bc4f-4fe9-9572-ac673f28a3be","Type":"ContainerStarted","Data":"3ec7abcdb58926c6133f14336f8c11fd81e2196219057269b10df3abbccc850a"}
Sep 17 23:21:40.801589 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.801549    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b" event={"ID":"9a4659d1-bc4f-4fe9-9572-ac673f28a3be","Type":"ContainerStarted","Data":"8402a80d6a68df78140d49fa9674035ad5b2a4b88921a7688eaa084328071fb9"}
Sep 17 23:21:40.801589 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.801572    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:40.819807 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.819761    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x" podStartSLOduration=1.4927020180000001 podStartE2EDuration="1.819746811s" totalImagesPullingTime="327.044793ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:21:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:21:40.817618546 +0000 UTC m=+924.998256986" watchObservedRunningTime="2026-09-17 23:21:40.819746811 +0000 UTC m=+925.000385240"
Sep 17 23:21:40.834470 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.834433    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b" podStartSLOduration=2.438762314 podStartE2EDuration="2.834419521s" totalImagesPullingTime="395.657207ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:21:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:21:40.833533752 +0000 UTC m=+925.014172184" watchObservedRunningTime="2026-09-17 23:21:40.834419521 +0000 UTC m=+925.015057959"
Sep 17 23:21:40.920109 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.920078    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"]
Sep 17 23:21:40.923415 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.923393    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:40.926243 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.926109    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-0c363f-serving-cert\""
Sep 17 23:21:40.934214 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.934192    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"]
Sep 17 23:21:40.957996 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:40.957975    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:41.049813 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.049772    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/734c919e-fa4b-422d-9c5a-10bc0b305c23-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:41.049971 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.049829    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-metrics-certs\") pod \"maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:41.049971 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.049884    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-maas-api-tls\") pod \"maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:41.049971 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.049964    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfx7z\" (UniqueName: \"kubernetes.io/projected/734c919e-fa4b-422d-9c5a-10bc0b305c23-kube-api-access-kfx7z\") pod \"maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:41.058394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.058315    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:41.150767 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.150737    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/734c919e-fa4b-422d-9c5a-10bc0b305c23-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:41.150885 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.150780    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-metrics-certs\") pod \"maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:41.150885 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.150804    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-maas-api-tls\") pod \"maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:41.150885 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.150861    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfx7z\" (UniqueName: \"kubernetes.io/projected/734c919e-fa4b-422d-9c5a-10bc0b305c23-kube-api-access-kfx7z\") pod \"maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:41.151064 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:21:41.150961    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-serving-cert: secret "maas-api-e2e-inf-a-0c363f-serving-cert" not found
Sep 17 23:21:41.151064 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:21:41.151031    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-maas-api-tls podName:734c919e-fa4b-422d-9c5a-10bc0b305c23 nodeName:}" failed. No retries permitted until 2026-09-17 23:21:41.651014476 +0000 UTC m=+925.831652906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-maas-api-tls") pod "maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf" (UID: "734c919e-fa4b-422d-9c5a-10bc0b305c23") : secret "maas-api-e2e-inf-a-0c363f-serving-cert" not found
Sep 17 23:21:41.151499 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.151473    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/734c919e-fa4b-422d-9c5a-10bc0b305c23-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:41.153298 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.153275    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-metrics-certs\") pod \"maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:41.159627 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.159603    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfx7z\" (UniqueName: \"kubernetes.io/projected/734c919e-fa4b-422d-9c5a-10bc0b305c23-kube-api-access-kfx7z\") pod \"maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:41.382382 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.382326    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:41.578901 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.578872    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:41.655911 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.655839    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-maas-api-tls\") pod \"maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:41.658455 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.658431    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-maas-api-tls\") pod \"maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:41.665820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.665796    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:41.836192 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:41.836165    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:42.066802 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:42.066772    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:42.105826 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:42.105796    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"]
Sep 17 23:21:42.107273 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:21:42.107250    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod734c919e_fa4b_422d_9c5a_10bc0b305c23.slice/crio-8305e78ff947c9436580982920b9511e96287f3c081182236e59010c8c4175c0 WatchSource:0}: Error finding container 8305e78ff947c9436580982920b9511e96287f3c081182236e59010c8c4175c0: Status 404 returned error can't find the container with id 8305e78ff947c9436580982920b9511e96287f3c081182236e59010c8c4175c0
Sep 17 23:21:42.761613 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:42.761572    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:42.814874 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:42.814844    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf" event={"ID":"734c919e-fa4b-422d-9c5a-10bc0b305c23","Type":"ContainerStarted","Data":"f5442455b6a773c72cc64719e7c6b133cb450d082153ea5769a9a321e93e3518"}
Sep 17 23:21:42.815011 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:42.814916    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf" event={"ID":"734c919e-fa4b-422d-9c5a-10bc0b305c23","Type":"ContainerStarted","Data":"8305e78ff947c9436580982920b9511e96287f3c081182236e59010c8c4175c0"}
Sep 17 23:21:42.815011 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:42.814991    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:42.835531 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:42.835488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:42.838816 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:42.838779    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf" podStartSLOduration=2.529100845 podStartE2EDuration="2.838766286s" totalImagesPullingTime="309.665441ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:21:42.837404415 +0000 UTC m=+927.018042852" watchObservedRunningTime="2026-09-17 23:21:42.838766286 +0000 UTC m=+927.019404724"
Sep 17 23:21:42.906224 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:42.906194    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:43.007981 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.007952    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:43.267523 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.267482    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"]
Sep 17 23:21:43.271297 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.271276    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.273624 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.273598    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-021e6520-openshift-default-dockercfg-cfbrc\""
Sep 17 23:21:43.284048 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.284025    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"]
Sep 17 23:21:43.296599 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.296577    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:43.371288 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.371258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51a49042-43e1-43de-a08f-6a7457d58bba-istio-token\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.371442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.371295    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-workload-certs\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.371442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.371313    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8bzw\" (UniqueName: \"kubernetes.io/projected/51a49042-43e1-43de-a08f-6a7457d58bba-kube-api-access-x8bzw\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.371442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.371387    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51a49042-43e1-43de-a08f-6a7457d58bba-istiod-ca-cert\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.371652 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.371463    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-istio-envoy\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.371652 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.371534    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-workload-socket\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.371652 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.371558    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51a49042-43e1-43de-a08f-6a7457d58bba-istio-podinfo\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.371652 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.371604    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-istio-data\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.371875 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.371677    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-credential-socket\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.472106 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.472074    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51a49042-43e1-43de-a08f-6a7457d58bba-istio-token\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.472106 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.472112    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-workload-certs\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.472297 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.472130    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8bzw\" (UniqueName: \"kubernetes.io/projected/51a49042-43e1-43de-a08f-6a7457d58bba-kube-api-access-x8bzw\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.472297 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.472160    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51a49042-43e1-43de-a08f-6a7457d58bba-istiod-ca-cert\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.472297 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.472206    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-istio-envoy\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.472297 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.472253    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-workload-socket\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.472297 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.472276    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51a49042-43e1-43de-a08f-6a7457d58bba-istio-podinfo\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.472559 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.472345    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-istio-data\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.472559 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.472387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-credential-socket\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.472694 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.472670    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-workload-certs\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.472854 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.472825    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-workload-socket\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.472951 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.472769    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-credential-socket\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.473102 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.473082    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-istio-data\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.473170 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.473129    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51a49042-43e1-43de-a08f-6a7457d58bba-istiod-ca-cert\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.474720 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.474696    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-istio-envoy\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.474787 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.474745    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51a49042-43e1-43de-a08f-6a7457d58bba-istio-podinfo\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.480240 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.480209    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51a49042-43e1-43de-a08f-6a7457d58bba-istio-token\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.480387 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.480365    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8bzw\" (UniqueName: \"kubernetes.io/projected/51a49042-43e1-43de-a08f-6a7457d58bba-kube-api-access-x8bzw\") pod \"e2e-ait-021e6520-openshift-default-5766dc66f4-42dln\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") " pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.581705 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.581643    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:43.674764 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.674731    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:43.779398 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.779366    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:43.858825 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.858798    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"]
Sep 17 23:21:43.860536 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:21:43.860487    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51a49042_43e1_43de_a08f_6a7457d58bba.slice/crio-dd84729adda3d13ea375765676f551f9136a86a0bce745643ba8156756f58cc4 WatchSource:0}: Error finding container dd84729adda3d13ea375765676f551f9136a86a0bce745643ba8156756f58cc4: Status 404 returned error can't find the container with id dd84729adda3d13ea375765676f551f9136a86a0bce745643ba8156756f58cc4
Sep 17 23:21:43.863055 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.863023    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:21:43.863158 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.863097    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:21:43.863158 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:43.863137    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:21:44.825851 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:44.825817    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln" event={"ID":"51a49042-43e1-43de-a08f-6a7457d58bba","Type":"ContainerStarted","Data":"26c8fe6b1a998cfb86f8531d26ccd7dec4d4f96c85722fd3502a978eb2547167"}
Sep 17 23:21:44.826009 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:44.825870    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln" event={"ID":"51a49042-43e1-43de-a08f-6a7457d58bba","Type":"ContainerStarted","Data":"dd84729adda3d13ea375765676f551f9136a86a0bce745643ba8156756f58cc4"}
Sep 17 23:21:44.849066 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:44.849022    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln" podStartSLOduration=1.849007431 podStartE2EDuration="1.849007431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:21:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:21:44.84792482 +0000 UTC m=+929.028563294" watchObservedRunningTime="2026-09-17 23:21:44.849007431 +0000 UTC m=+929.029645869"
Sep 17 23:21:45.582056 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:45.582016    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:45.586881 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:45.586856    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:45.608267 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:45.608241    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:45.831733 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:45.831703    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:45.832487 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:45.832433    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:21:46.270197 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:46.270157    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:46.375347 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:46.375318    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:46.814138 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:46.814106    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:21:46.814571 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:46.814477    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:21:46.863084 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:46.863047    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:48.134741 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:48.134711    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:48.290116 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:48.290080    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:48.455771 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:48.455742    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:48.528761 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:48.528725    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:48.824943 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:48.824858    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:21:50.117338 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:50.117307    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:51.958130 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:51.958093    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:52.035016 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.034981    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"]
Sep 17 23:21:52.038833 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.038816    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.042489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.042469    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-021e6520-serving-cert\""
Sep 17 23:21:52.052758 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.052731    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"]
Sep 17 23:21:52.058841 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.058814    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc2bl\" (UniqueName: \"kubernetes.io/projected/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-kube-api-access-hc2bl\") pod \"maas-api-e2e-ait-021e6520-b556c947f-2rpbn\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.058973 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.058845    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-metrics-certs\") pod \"maas-api-e2e-ait-021e6520-b556c947f-2rpbn\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.058973 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.058895    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-openshift-service-ca\") pod \"maas-api-e2e-ait-021e6520-b556c947f-2rpbn\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.058973 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.058940    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-maas-api-tls\") pod \"maas-api-e2e-ait-021e6520-b556c947f-2rpbn\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.060557 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.060531    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:52.136029 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.135998    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:52.159983 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.159949    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-maas-api-tls\") pod \"maas-api-e2e-ait-021e6520-b556c947f-2rpbn\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.160149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.160021    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc2bl\" (UniqueName: \"kubernetes.io/projected/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-kube-api-access-hc2bl\") pod \"maas-api-e2e-ait-021e6520-b556c947f-2rpbn\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.160149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.160050    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-metrics-certs\") pod \"maas-api-e2e-ait-021e6520-b556c947f-2rpbn\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.160149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.160108    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-openshift-service-ca\") pod \"maas-api-e2e-ait-021e6520-b556c947f-2rpbn\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.160824 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.160798    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-openshift-service-ca\") pod \"maas-api-e2e-ait-021e6520-b556c947f-2rpbn\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.163199 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.163176    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-maas-api-tls\") pod \"maas-api-e2e-ait-021e6520-b556c947f-2rpbn\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.163300 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.163220    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-metrics-certs\") pod \"maas-api-e2e-ait-021e6520-b556c947f-2rpbn\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.167959 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.167937    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc2bl\" (UniqueName: \"kubernetes.io/projected/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-kube-api-access-hc2bl\") pod \"maas-api-e2e-ait-021e6520-b556c947f-2rpbn\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.238331 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.238224    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:52.339619 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.339582    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:52.353776 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.353745    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:52.645752 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.645721    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"]
Sep 17 23:21:52.647629 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:21:52.647608    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c46eec4_f801_4a06_9ce4_9e4629c9b0eb.slice/crio-a26abd98b522df8b41a289c17a16009f021566bdebf4cc256e1c361c0448fdb8 WatchSource:0}: Error finding container a26abd98b522df8b41a289c17a16009f021566bdebf4cc256e1c361c0448fdb8: Status 404 returned error can't find the container with id a26abd98b522df8b41a289c17a16009f021566bdebf4cc256e1c361c0448fdb8
Sep 17 23:21:52.667987 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.667951    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:52.771239 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.771184    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:52.869924 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.869888    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn" event={"ID":"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb","Type":"ContainerStarted","Data":"a26abd98b522df8b41a289c17a16009f021566bdebf4cc256e1c361c0448fdb8"}
Sep 17 23:21:52.930792 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:52.930721    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:53.096461 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:53.096419    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:53.200045 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:53.200007    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:53.415384 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:53.415349    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:53.796709 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:53.796666    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:53.877411 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:53.877376    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn" event={"ID":"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb","Type":"ContainerStarted","Data":"216d8bff65946ab67b3db1e98348bcd0d5a53c3a8c26be26879b873eb18a2b06"}
Sep 17 23:21:53.877573 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:53.877442    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:53.897266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:53.897212    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn" podStartSLOduration=1.591117042 podStartE2EDuration="1.897195441s" totalImagesPullingTime="306.078399ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:21:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:21:53.895270295 +0000 UTC m=+938.075908732" watchObservedRunningTime="2026-09-17 23:21:53.897195441 +0000 UTC m=+938.077833879"
Sep 17 23:21:54.618327 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:54.618295    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:54.769529 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:54.769473    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:54.918397 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:54.918293    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:55.020630 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:55.020602    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:55.116883 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:55.116842    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:55.515194 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:55.515152    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:56.332277 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.332243    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"]
Sep 17 23:21:56.336262 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.336238    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:56.338615 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.338592    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-63b144-serving-cert\""
Sep 17 23:21:56.352042 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.352018    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"]
Sep 17 23:21:56.373585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.373555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:56.404719 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.404680    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt8x6\" (UniqueName: \"kubernetes.io/projected/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-kube-api-access-tt8x6\") pod \"maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:56.404891 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.404781    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-metrics-certs\") pod \"maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:56.404891 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.404817    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-openshift-service-ca\") pod \"maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:56.404891 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.404859    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-maas-api-tls\") pod \"maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:56.432894 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.432865    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:56.505436 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.505388    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-metrics-certs\") pod \"maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:56.505643 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.505450    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-openshift-service-ca\") pod \"maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:56.505643 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.505503    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-maas-api-tls\") pod \"maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:56.505643 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.505604    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt8x6\" (UniqueName: \"kubernetes.io/projected/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-kube-api-access-tt8x6\") pod \"maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:56.505643 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:21:56.505629    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-63b144-serving-cert: secret "maas-api-e2e-autores-63b144-serving-cert" not found
Sep 17 23:21:56.505800 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:21:56.505705    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-maas-api-tls podName:de0c4bff-b9bd-4b7c-a1cc-ea50e863619d nodeName:}" failed. No retries permitted until 2026-09-17 23:21:57.005681594 +0000 UTC m=+941.186320010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-maas-api-tls") pod "maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk" (UID: "de0c4bff-b9bd-4b7c-a1cc-ea50e863619d") : secret "maas-api-e2e-autores-63b144-serving-cert" not found
Sep 17 23:21:56.506191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.506165    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-openshift-service-ca\") pod \"maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:56.508122 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.508094    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-metrics-certs\") pod \"maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:56.513719 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.513697    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt8x6\" (UniqueName: \"kubernetes.io/projected/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-kube-api-access-tt8x6\") pod \"maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:56.623600 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.623490    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:56.786357 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.786326    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:56.880341 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.880264    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:56.992488 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:56.992448    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:57.010762 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:57.010736    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-maas-api-tls\") pod \"maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:57.013131 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:57.013111    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-maas-api-tls\") pod \"maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:57.191938 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:57.191899    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:57.247233 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:57.247195    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:57.523034 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:57.523009    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"]
Sep 17 23:21:57.524888 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:21:57.524857    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde0c4bff_b9bd_4b7c_a1cc_ea50e863619d.slice/crio-bffb251837c41f6c31e37c893b09481c7f0674e38b6920c0e43249324fc6b723 WatchSource:0}: Error finding container bffb251837c41f6c31e37c893b09481c7f0674e38b6920c0e43249324fc6b723: Status 404 returned error can't find the container with id bffb251837c41f6c31e37c893b09481c7f0674e38b6920c0e43249324fc6b723
Sep 17 23:21:57.609121 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:57.609087    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:57.839242 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:57.839167    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:57.906068 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:57.906025    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk" event={"ID":"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d","Type":"ContainerStarted","Data":"bffb251837c41f6c31e37c893b09481c7f0674e38b6920c0e43249324fc6b723"}
Sep 17 23:21:58.630027 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:58.629994    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:58.913797 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:58.913710    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk" event={"ID":"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d","Type":"ContainerStarted","Data":"c0dfc9ca755a0bbc3391cf117196cc0357a1fd288b5af69c3c765189268e6ddc"}
Sep 17 23:21:58.913926 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:58.913885    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:21:58.934130 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:58.934083    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk" podStartSLOduration=2.580295606 podStartE2EDuration="2.93406867s" totalImagesPullingTime="353.773064ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:21:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:21:58.931327882 +0000 UTC m=+943.111966320" watchObservedRunningTime="2026-09-17 23:21:58.93406867 +0000 UTC m=+943.114707139"
Sep 17 23:21:59.496467 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.496431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:59.785104 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.785013    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"]
Sep 17 23:21:59.789250 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.789221    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:21:59.791661 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.791637    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-307a7b-serving-cert\""
Sep 17 23:21:59.800063 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.800031    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"]
Sep 17 23:21:59.834694 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.834664    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:59.837880 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.837860    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35bc8078-6843-4901-8a3c-e5d4672dc763-openshift-service-ca\") pod \"maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:21:59.837960 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.837929    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-maas-api-tls\") pod \"maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:21:59.837960 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.837947    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqslm\" (UniqueName: \"kubernetes.io/projected/35bc8078-6843-4901-8a3c-e5d4672dc763-kube-api-access-sqslm\") pod \"maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:21:59.838050 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.837966    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-metrics-certs\") pod \"maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:21:59.891646 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.891613    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:21:59.918498 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.918468    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:21:59.939535 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.938386    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-maas-api-tls\") pod \"maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:21:59.939535 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.938418    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqslm\" (UniqueName: \"kubernetes.io/projected/35bc8078-6843-4901-8a3c-e5d4672dc763-kube-api-access-sqslm\") pod \"maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:21:59.939535 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.938448    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-metrics-certs\") pod \"maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:21:59.939535 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:21:59.938568    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-serving-cert: secret "maas-api-e2e-inf-b-307a7b-serving-cert" not found
Sep 17 23:21:59.939535 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:21:59.938657    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-maas-api-tls podName:35bc8078-6843-4901-8a3c-e5d4672dc763 nodeName:}" failed. No retries permitted until 2026-09-17 23:22:00.438634073 +0000 UTC m=+944.619272507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-maas-api-tls") pod "maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct" (UID: "35bc8078-6843-4901-8a3c-e5d4672dc763") : secret "maas-api-e2e-inf-b-307a7b-serving-cert" not found
Sep 17 23:21:59.939535 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.939256    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35bc8078-6843-4901-8a3c-e5d4672dc763-openshift-service-ca\") pod \"maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:21:59.940079 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.940052    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35bc8078-6843-4901-8a3c-e5d4672dc763-openshift-service-ca\") pod \"maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:21:59.942298 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.942272    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-metrics-certs\") pod \"maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:21:59.970587 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.970554    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqslm\" (UniqueName: \"kubernetes.io/projected/35bc8078-6843-4901-8a3c-e5d4672dc763-kube-api-access-sqslm\") pod \"maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:21:59.986117 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:21:59.986091    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:00.444781 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:00.444747    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-maas-api-tls\") pod \"maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:22:00.448101 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:00.448072    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-maas-api-tls\") pod \"maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:22:00.465460 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:00.465430    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:00.563445 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:00.563392    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:00.703816 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:00.703733    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:22:01.011641 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:01.011612    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"]
Sep 17 23:22:01.013453 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:22:01.013430    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35bc8078_6843_4901_8a3c_e5d4672dc763.slice/crio-46e893f7f5528e81393a546c5a2efe93f81c66a3d2525afeb9221d60cffa9304 WatchSource:0}: Error finding container 46e893f7f5528e81393a546c5a2efe93f81c66a3d2525afeb9221d60cffa9304: Status 404 returned error can't find the container with id 46e893f7f5528e81393a546c5a2efe93f81c66a3d2525afeb9221d60cffa9304
Sep 17 23:22:01.155988 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:01.155943    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:01.937340 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:01.937308    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct" event={"ID":"35bc8078-6843-4901-8a3c-e5d4672dc763","Type":"ContainerStarted","Data":"de0e38351e64bbb27aa43e692a10cee543879075c38bb30d58da71045a7cdf59"}
Sep 17 23:22:01.937558 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:01.937355    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct" event={"ID":"35bc8078-6843-4901-8a3c-e5d4672dc763","Type":"ContainerStarted","Data":"46e893f7f5528e81393a546c5a2efe93f81c66a3d2525afeb9221d60cffa9304"}
Sep 17 23:22:01.937558 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:01.937393    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:22:01.957679 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:01.957619    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct" podStartSLOduration=2.597647166 podStartE2EDuration="2.957596765s" totalImagesPullingTime="359.949599ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:21:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:22:01.95272975 +0000 UTC m=+946.133368193" watchObservedRunningTime="2026-09-17 23:22:01.957596765 +0000 UTC m=+946.138235204"
Sep 17 23:22:02.392113 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:02.392012    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:03.529574 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:03.529536    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:04.796968 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:04.796925    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:04.931021 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:04.930986    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:22:05.514574 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:05.514508    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:05.600625 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:05.600592    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:05.743423 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:05.743394    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:06.304216 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:06.304154    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:06.797485 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:06.797450    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:06.966583 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:06.966546    2558 generic.go:356] "Generic (PLEG): container finished" podID="51a49042-43e1-43de-a08f-6a7457d58bba" containerID="26c8fe6b1a998cfb86f8531d26ccd7dec4d4f96c85722fd3502a978eb2547167" exitCode=0
Sep 17 23:22:06.966781 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:06.966590    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln" event={"ID":"51a49042-43e1-43de-a08f-6a7457d58bba","Type":"ContainerDied","Data":"26c8fe6b1a998cfb86f8531d26ccd7dec4d4f96c85722fd3502a978eb2547167"}
Sep 17 23:22:06.967052 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:06.967034    2558 scope.go:117] "RemoveContainer" containerID="26c8fe6b1a998cfb86f8531d26ccd7dec4d4f96c85722fd3502a978eb2547167"
Sep 17 23:22:06.968210 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:06.968178    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:22:06.968306 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:06.968264    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:22:06.968379 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:06.968315    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:22:07.648153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:07.648115    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:07.953682 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:07.952785    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:22:07.977813 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:07.977770    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln" event={"ID":"51a49042-43e1-43de-a08f-6a7457d58bba","Type":"ContainerStarted","Data":"029aea00bc653514ca0d87f173205073576baca289c4510905a678004e0f0298"}
Sep 17 23:22:08.582402 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:08.582362    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:22:08.582741 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:08.582686    2558 patch_prober.go:28] interesting pod/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 17 23:22:08.582831 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:08.582760    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln" podUID="51a49042-43e1-43de-a08f-6a7457d58bba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 17 23:22:08.675563 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:08.675507    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:09.576736 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:09.576694    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:09.582843 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:09.582824    2558 patch_prober.go:28] interesting pod/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 17 23:22:09.582925 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:09.582868    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln" podUID="51a49042-43e1-43de-a08f-6a7457d58bba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 17 23:22:10.582279 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:10.582239    2558 patch_prober.go:28] interesting pod/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 17 23:22:10.582697 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:10.582299    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln" podUID="51a49042-43e1-43de-a08f-6a7457d58bba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 17 23:22:10.817482 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:10.817444    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:11.582200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:11.582157    2558 patch_prober.go:28] interesting pod/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 17 23:22:11.582392 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:11.582224    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln" podUID="51a49042-43e1-43de-a08f-6a7457d58bba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 17 23:22:11.695283 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:11.695253    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:11.893168 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:11.893075    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:11.991634 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:11.991600    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:12.088202 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:12.088168    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:12.247304 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:12.247266    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:12.582830 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:12.582737    2558 patch_prober.go:28] interesting pod/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 17 23:22:12.582830 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:12.582794    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln" podUID="51a49042-43e1-43de-a08f-6a7457d58bba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 17 23:22:12.989942 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:12.989906    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:13.586291 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:13.586261    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:22:13.586783 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:13.586674    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:22:13.587335 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:13.587318    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:22:14.036050 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:14.036019    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:14.784094 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:14.784057    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:15.775779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:15.775730    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:16.540263 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:16.540221    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:16.640656 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:16.640584    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:17.476711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:17.476677    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:18.280607 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:18.280570    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:18.906208 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:18.906171    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:18.988268 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:18.988237    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:19.370763 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:19.370711    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:20.346278 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:20.346244    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:21.247663 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:21.247627    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:22.430399 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:22.430352    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:27.695417 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:27.695382    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:28.280197 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.280160    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"]
Sep 17 23:22:28.280479 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.280453    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn" podUID="2c46eec4-f801-4a06-9ce4-9e4629c9b0eb" containerName="maas-api" containerID="cri-o://216d8bff65946ab67b3db1e98348bcd0d5a53c3a8c26be26879b873eb18a2b06" gracePeriod=30
Sep 17 23:22:28.308628 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.308601    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:28.391145 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.391110    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:28.489789 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.489742    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:28.529303 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.529280    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:22:28.555115 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.554859    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:28.634787 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.634753    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-maas-api-tls\") pod \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") "
Sep 17 23:22:28.635000 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.634810    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc2bl\" (UniqueName: \"kubernetes.io/projected/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-kube-api-access-hc2bl\") pod \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") "
Sep 17 23:22:28.635000 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.634834    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-metrics-certs\") pod \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") "
Sep 17 23:22:28.635000 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.634929    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-openshift-service-ca\") pod \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\" (UID: \"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb\") "
Sep 17 23:22:28.635364 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.635332    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2c46eec4-f801-4a06-9ce4-9e4629c9b0eb" (UID: "2c46eec4-f801-4a06-9ce4-9e4629c9b0eb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:22:28.637030 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.637005    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2c46eec4-f801-4a06-9ce4-9e4629c9b0eb" (UID: "2c46eec4-f801-4a06-9ce4-9e4629c9b0eb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:22:28.637124 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.637026    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2c46eec4-f801-4a06-9ce4-9e4629c9b0eb" (UID: "2c46eec4-f801-4a06-9ce4-9e4629c9b0eb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:22:28.637124 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.637052    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-kube-api-access-hc2bl" (OuterVolumeSpecName: "kube-api-access-hc2bl") pod "2c46eec4-f801-4a06-9ce4-9e4629c9b0eb" (UID: "2c46eec4-f801-4a06-9ce4-9e4629c9b0eb"). InnerVolumeSpecName "kube-api-access-hc2bl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:22:28.690935 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.690885    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"]
Sep 17 23:22:28.691135 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.691107    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln" podUID="51a49042-43e1-43de-a08f-6a7457d58bba" containerName="istio-proxy" containerID="cri-o://029aea00bc653514ca0d87f173205073576baca289c4510905a678004e0f0298" gracePeriod=30
Sep 17 23:22:28.699976 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.699947    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:28.736008 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.735973    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:22:28.736008 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.736001    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:22:28.736008 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.736013    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc2bl\" (UniqueName: \"kubernetes.io/projected/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-kube-api-access-hc2bl\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:22:28.736008 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.736021    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:22:28.997390 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:28.997355    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:29.095746 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.095713    2558 generic.go:356] "Generic (PLEG): container finished" podID="2c46eec4-f801-4a06-9ce4-9e4629c9b0eb" containerID="216d8bff65946ab67b3db1e98348bcd0d5a53c3a8c26be26879b873eb18a2b06" exitCode=0
Sep 17 23:22:29.095950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.095768    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"
Sep 17 23:22:29.095950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.095795    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn" event={"ID":"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb","Type":"ContainerDied","Data":"216d8bff65946ab67b3db1e98348bcd0d5a53c3a8c26be26879b873eb18a2b06"}
Sep 17 23:22:29.095950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.095853    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn" event={"ID":"2c46eec4-f801-4a06-9ce4-9e4629c9b0eb","Type":"ContainerDied","Data":"a26abd98b522df8b41a289c17a16009f021566bdebf4cc256e1c361c0448fdb8"}
Sep 17 23:22:29.095950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.095878    2558 scope.go:117] "RemoveContainer" containerID="216d8bff65946ab67b3db1e98348bcd0d5a53c3a8c26be26879b873eb18a2b06"
Sep 17 23:22:29.105946 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.105927    2558 scope.go:117] "RemoveContainer" containerID="216d8bff65946ab67b3db1e98348bcd0d5a53c3a8c26be26879b873eb18a2b06"
Sep 17 23:22:29.106270 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:22:29.106251    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"216d8bff65946ab67b3db1e98348bcd0d5a53c3a8c26be26879b873eb18a2b06\": container with ID starting with 216d8bff65946ab67b3db1e98348bcd0d5a53c3a8c26be26879b873eb18a2b06 not found: ID does not exist" containerID="216d8bff65946ab67b3db1e98348bcd0d5a53c3a8c26be26879b873eb18a2b06"
Sep 17 23:22:29.106325 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.106280    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"216d8bff65946ab67b3db1e98348bcd0d5a53c3a8c26be26879b873eb18a2b06"} err="failed to get container status \"216d8bff65946ab67b3db1e98348bcd0d5a53c3a8c26be26879b873eb18a2b06\": rpc error: code = NotFound desc = could not find container \"216d8bff65946ab67b3db1e98348bcd0d5a53c3a8c26be26879b873eb18a2b06\": container with ID starting with 216d8bff65946ab67b3db1e98348bcd0d5a53c3a8c26be26879b873eb18a2b06 not found: ID does not exist"
Sep 17 23:22:29.118853 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.118821    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"]
Sep 17 23:22:29.122493 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.122470    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-021e6520-b556c947f-2rpbn"]
Sep 17 23:22:29.212096 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.212060    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:29.299038 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.298956    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:29.417955 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.417921    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:29.834851 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.834822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:29.940444 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:29.940394    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c46eec4-f801-4a06-9ce4-9e4629c9b0eb" path="/var/lib/kubelet/pods/2c46eec4-f801-4a06-9ce4-9e4629c9b0eb/volumes"
Sep 17 23:22:30.073328 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:30.073245    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:30.462368 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:30.462337    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:30.771629 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:30.771539    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:30.831400 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:30.831370    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:30.964474 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:30.964436    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:31.859838 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:31.859803    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:32.534050 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:32.534015    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:32.627353 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:32.627320    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:33.336088 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.336046    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:33.944055 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.944032    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:22:33.985119 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.985040    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-istio-envoy\") pod \"51a49042-43e1-43de-a08f-6a7457d58bba\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") "
Sep 17 23:22:33.985119 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.985070    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51a49042-43e1-43de-a08f-6a7457d58bba-istio-podinfo\") pod \"51a49042-43e1-43de-a08f-6a7457d58bba\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") "
Sep 17 23:22:33.985119 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.985088    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-credential-socket\") pod \"51a49042-43e1-43de-a08f-6a7457d58bba\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") "
Sep 17 23:22:33.985119 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.985114    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-istio-data\") pod \"51a49042-43e1-43de-a08f-6a7457d58bba\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") "
Sep 17 23:22:33.985423 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.985162    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-workload-socket\") pod \"51a49042-43e1-43de-a08f-6a7457d58bba\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") "
Sep 17 23:22:33.985423 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.985214    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51a49042-43e1-43de-a08f-6a7457d58bba-istio-token\") pod \"51a49042-43e1-43de-a08f-6a7457d58bba\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") "
Sep 17 23:22:33.985423 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.985231    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51a49042-43e1-43de-a08f-6a7457d58bba-istiod-ca-cert\") pod \"51a49042-43e1-43de-a08f-6a7457d58bba\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") "
Sep 17 23:22:33.985423 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.985299    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8bzw\" (UniqueName: \"kubernetes.io/projected/51a49042-43e1-43de-a08f-6a7457d58bba-kube-api-access-x8bzw\") pod \"51a49042-43e1-43de-a08f-6a7457d58bba\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") "
Sep 17 23:22:33.985423 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.985350    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-workload-certs\") pod \"51a49042-43e1-43de-a08f-6a7457d58bba\" (UID: \"51a49042-43e1-43de-a08f-6a7457d58bba\") "
Sep 17 23:22:33.985806 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.985779    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "51a49042-43e1-43de-a08f-6a7457d58bba" (UID: "51a49042-43e1-43de-a08f-6a7457d58bba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:22:33.985892 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.985856    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "51a49042-43e1-43de-a08f-6a7457d58bba" (UID: "51a49042-43e1-43de-a08f-6a7457d58bba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:22:33.985962 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.985880    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51a49042-43e1-43de-a08f-6a7457d58bba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "51a49042-43e1-43de-a08f-6a7457d58bba" (UID: "51a49042-43e1-43de-a08f-6a7457d58bba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:22:33.986100 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.986063    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "51a49042-43e1-43de-a08f-6a7457d58bba" (UID: "51a49042-43e1-43de-a08f-6a7457d58bba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:22:33.986184 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.986158    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-istio-data" (OuterVolumeSpecName: "istio-data") pod "51a49042-43e1-43de-a08f-6a7457d58bba" (UID: "51a49042-43e1-43de-a08f-6a7457d58bba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:22:33.987843 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.987809    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51a49042-43e1-43de-a08f-6a7457d58bba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "51a49042-43e1-43de-a08f-6a7457d58bba" (UID: "51a49042-43e1-43de-a08f-6a7457d58bba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:22:33.987945 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.987848    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "51a49042-43e1-43de-a08f-6a7457d58bba" (UID: "51a49042-43e1-43de-a08f-6a7457d58bba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:22:33.987995 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.987972    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51a49042-43e1-43de-a08f-6a7457d58bba-istio-token" (OuterVolumeSpecName: "istio-token") pod "51a49042-43e1-43de-a08f-6a7457d58bba" (UID: "51a49042-43e1-43de-a08f-6a7457d58bba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:22:33.988253 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:33.988230    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51a49042-43e1-43de-a08f-6a7457d58bba-kube-api-access-x8bzw" (OuterVolumeSpecName: "kube-api-access-x8bzw") pod "51a49042-43e1-43de-a08f-6a7457d58bba" (UID: "51a49042-43e1-43de-a08f-6a7457d58bba"). InnerVolumeSpecName "kube-api-access-x8bzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:22:34.086932 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.086897    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8bzw\" (UniqueName: \"kubernetes.io/projected/51a49042-43e1-43de-a08f-6a7457d58bba-kube-api-access-x8bzw\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:22:34.086932 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.086924    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-workload-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:22:34.086932 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.086934    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-istio-envoy\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:22:34.087162 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.086944    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51a49042-43e1-43de-a08f-6a7457d58bba-istio-podinfo\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:22:34.087162 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.086951    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-credential-socket\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:22:34.087162 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.086959    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-istio-data\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:22:34.087162 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.086967    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51a49042-43e1-43de-a08f-6a7457d58bba-workload-socket\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:22:34.087162 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.086975    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51a49042-43e1-43de-a08f-6a7457d58bba-istio-token\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:22:34.087162 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.086983    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51a49042-43e1-43de-a08f-6a7457d58bba-istiod-ca-cert\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:22:34.125481 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.125446    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"]
Sep 17 23:22:34.126637 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.126616    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c46eec4-f801-4a06-9ce4-9e4629c9b0eb" containerName="maas-api"
Sep 17 23:22:34.126791 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.126780    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c46eec4-f801-4a06-9ce4-9e4629c9b0eb" containerName="maas-api"
Sep 17 23:22:34.126908 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.126898    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51a49042-43e1-43de-a08f-6a7457d58bba" containerName="istio-proxy"
Sep 17 23:22:34.126979 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.126970    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="51a49042-43e1-43de-a08f-6a7457d58bba" containerName="istio-proxy"
Sep 17 23:22:34.127261 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.127244    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="51a49042-43e1-43de-a08f-6a7457d58bba" containerName="istio-proxy"
Sep 17 23:22:34.127369 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.127358    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c46eec4-f801-4a06-9ce4-9e4629c9b0eb" containerName="maas-api"
Sep 17 23:22:34.127462 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.127451    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="51a49042-43e1-43de-a08f-6a7457d58bba" containerName="istio-proxy"
Sep 17 23:22:34.132101 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.132084    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.139865 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.139817    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"]
Sep 17 23:22:34.142641 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.140288    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-d683cd6a-serving-cert\""
Sep 17 23:22:34.149397 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.149369    2558 generic.go:356] "Generic (PLEG): container finished" podID="51a49042-43e1-43de-a08f-6a7457d58bba" containerID="029aea00bc653514ca0d87f173205073576baca289c4510905a678004e0f0298" exitCode=0
Sep 17 23:22:34.149548 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.149500    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln" event={"ID":"51a49042-43e1-43de-a08f-6a7457d58bba","Type":"ContainerDied","Data":"029aea00bc653514ca0d87f173205073576baca289c4510905a678004e0f0298"}
Sep 17 23:22:34.149633 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.149583    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln" event={"ID":"51a49042-43e1-43de-a08f-6a7457d58bba","Type":"ContainerDied","Data":"dd84729adda3d13ea375765676f551f9136a86a0bce745643ba8156756f58cc4"}
Sep 17 23:22:34.149633 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.149618    2558 scope.go:117] "RemoveContainer" containerID="029aea00bc653514ca0d87f173205073576baca289c4510905a678004e0f0298"
Sep 17 23:22:34.149748 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.149649    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"
Sep 17 23:22:34.164054 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.164030    2558 scope.go:117] "RemoveContainer" containerID="26c8fe6b1a998cfb86f8531d26ccd7dec4d4f96c85722fd3502a978eb2547167"
Sep 17 23:22:34.166571 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.166547    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:34.175282 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.175261    2558 scope.go:117] "RemoveContainer" containerID="029aea00bc653514ca0d87f173205073576baca289c4510905a678004e0f0298"
Sep 17 23:22:34.175586 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:22:34.175564    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"029aea00bc653514ca0d87f173205073576baca289c4510905a678004e0f0298\": container with ID starting with 029aea00bc653514ca0d87f173205073576baca289c4510905a678004e0f0298 not found: ID does not exist" containerID="029aea00bc653514ca0d87f173205073576baca289c4510905a678004e0f0298"
Sep 17 23:22:34.175669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.175597    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"029aea00bc653514ca0d87f173205073576baca289c4510905a678004e0f0298"} err="failed to get container status \"029aea00bc653514ca0d87f173205073576baca289c4510905a678004e0f0298\": rpc error: code = NotFound desc = could not find container \"029aea00bc653514ca0d87f173205073576baca289c4510905a678004e0f0298\": container with ID starting with 029aea00bc653514ca0d87f173205073576baca289c4510905a678004e0f0298 not found: ID does not exist"
Sep 17 23:22:34.175669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.175621    2558 scope.go:117] "RemoveContainer" containerID="26c8fe6b1a998cfb86f8531d26ccd7dec4d4f96c85722fd3502a978eb2547167"
Sep 17 23:22:34.175897 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:22:34.175880    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26c8fe6b1a998cfb86f8531d26ccd7dec4d4f96c85722fd3502a978eb2547167\": container with ID starting with 26c8fe6b1a998cfb86f8531d26ccd7dec4d4f96c85722fd3502a978eb2547167 not found: ID does not exist" containerID="26c8fe6b1a998cfb86f8531d26ccd7dec4d4f96c85722fd3502a978eb2547167"
Sep 17 23:22:34.175944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.175902    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26c8fe6b1a998cfb86f8531d26ccd7dec4d4f96c85722fd3502a978eb2547167"} err="failed to get container status \"26c8fe6b1a998cfb86f8531d26ccd7dec4d4f96c85722fd3502a978eb2547167\": rpc error: code = NotFound desc = could not find container \"26c8fe6b1a998cfb86f8531d26ccd7dec4d4f96c85722fd3502a978eb2547167\": container with ID starting with 26c8fe6b1a998cfb86f8531d26ccd7dec4d4f96c85722fd3502a978eb2547167 not found: ID does not exist"
Sep 17 23:22:34.179730 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.179711    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"]
Sep 17 23:22:34.184622 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.184602    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-021e6520-openshift-default-5766dc66f4-42dln"]
Sep 17 23:22:34.188335 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.188312    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-maas-api-tls\") pod \"maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.188430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.188356    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d857bf2-535e-4af3-8822-32b6cb8b19bb-openshift-service-ca\") pod \"maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.188430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.188398    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndh2h\" (UniqueName: \"kubernetes.io/projected/6d857bf2-535e-4af3-8822-32b6cb8b19bb-kube-api-access-ndh2h\") pod \"maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.188535 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.188477    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-metrics-certs\") pod \"maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.236119 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.236021    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:34.289943 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.289905    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d857bf2-535e-4af3-8822-32b6cb8b19bb-openshift-service-ca\") pod \"maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.290110 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.289953    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndh2h\" (UniqueName: \"kubernetes.io/projected/6d857bf2-535e-4af3-8822-32b6cb8b19bb-kube-api-access-ndh2h\") pod \"maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.290110 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.290015    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-metrics-certs\") pod \"maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.290266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.290107    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-maas-api-tls\") pod \"maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.290266 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:22:34.290212    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-serving-cert: secret "maas-api-e2e-migrate-d683cd6a-serving-cert" not found
Sep 17 23:22:34.290372 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:22:34.290281    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-maas-api-tls podName:6d857bf2-535e-4af3-8822-32b6cb8b19bb nodeName:}" failed. No retries permitted until 2026-09-17 23:22:34.790260326 +0000 UTC m=+978.970898759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-maas-api-tls") pod "maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj" (UID: "6d857bf2-535e-4af3-8822-32b6cb8b19bb") : secret "maas-api-e2e-migrate-d683cd6a-serving-cert" not found
Sep 17 23:22:34.290672 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.290647    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d857bf2-535e-4af3-8822-32b6cb8b19bb-openshift-service-ca\") pod \"maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.292429 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.292410    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-metrics-certs\") pod \"maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.300231 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.300212    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndh2h\" (UniqueName: \"kubernetes.io/projected/6d857bf2-535e-4af3-8822-32b6cb8b19bb-kube-api-access-ndh2h\") pod \"maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.461783 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.461749    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:34.782022 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.781984    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:34.795258 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.795225    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-maas-api-tls\") pod \"maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.797711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.797691    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-maas-api-tls\") pod \"maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:34.888714 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:34.888658    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:35.056084 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:35.055996    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:35.267200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:35.267166    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:35.341101 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:35.341070    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"]
Sep 17 23:22:35.342733 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:22:35.342702    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d857bf2_535e_4af3_8822_32b6cb8b19bb.slice/crio-a4b214a545564dbf2b9c97c3426ed8814a41f606515f4b66a8fc954447ca8a6d WatchSource:0}: Error finding container a4b214a545564dbf2b9c97c3426ed8814a41f606515f4b66a8fc954447ca8a6d: Status 404 returned error can't find the container with id a4b214a545564dbf2b9c97c3426ed8814a41f606515f4b66a8fc954447ca8a6d
Sep 17 23:22:35.931930 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:35.931852    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51a49042-43e1-43de-a08f-6a7457d58bba" path="/var/lib/kubelet/pods/51a49042-43e1-43de-a08f-6a7457d58bba/volumes"
Sep 17 23:22:36.068016 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:36.067978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:36.162594 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:36.162559    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj" event={"ID":"6d857bf2-535e-4af3-8822-32b6cb8b19bb","Type":"ContainerStarted","Data":"5e6773de839fa88895d351f2d7e4d99b5b01f59fedcffd195a29fb8020cc7acd"}
Sep 17 23:22:36.162768 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:36.162616    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj" event={"ID":"6d857bf2-535e-4af3-8822-32b6cb8b19bb","Type":"ContainerStarted","Data":"a4b214a545564dbf2b9c97c3426ed8814a41f606515f4b66a8fc954447ca8a6d"}
Sep 17 23:22:36.162768 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:36.162663    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:36.183221 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:36.183135    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj" podStartSLOduration=1.862620505 podStartE2EDuration="2.183119696s" totalImagesPullingTime="320.499191ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:22:36.180585627 +0000 UTC m=+980.361224070" watchObservedRunningTime="2026-09-17 23:22:36.183119696 +0000 UTC m=+980.363758158"
Sep 17 23:22:36.839658 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:36.839627    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:37.604105 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:37.604075    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:42.172584 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:42.172551    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:22:50.448232 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:50.448193    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:50.548821 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:50.548789    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:56.760182 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:56.760143    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:57.840539 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:57.840476    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:58.583324 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:58.583293    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:22:59.463524 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:22:59.463481    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:00.157457 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:00.157412    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:00.947128 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:00.947100    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:01.754179 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:01.754135    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:02.463838 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:02.463807    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:03.282971 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:03.282940    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:03.878054 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:03.878019    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"]
Sep 17 23:23:03.878449 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:03.878265    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj" podUID="6d857bf2-535e-4af3-8822-32b6cb8b19bb" containerName="maas-api" containerID="cri-o://5e6773de839fa88895d351f2d7e4d99b5b01f59fedcffd195a29fb8020cc7acd" gracePeriod=30
Sep 17 23:23:03.913817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:03.913789    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:03.982770 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:03.982742    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:04.042733 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.042695    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:04.112891 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.112867    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:04.118597 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.118575    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:23:04.176745 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.176680    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:04.242655 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.242614    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-metrics-certs\") pod \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") "
Sep 17 23:23:04.242810 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.242713    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d857bf2-535e-4af3-8822-32b6cb8b19bb-openshift-service-ca\") pod \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") "
Sep 17 23:23:04.242810 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.242788    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndh2h\" (UniqueName: \"kubernetes.io/projected/6d857bf2-535e-4af3-8822-32b6cb8b19bb-kube-api-access-ndh2h\") pod \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") "
Sep 17 23:23:04.242933 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.242817    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-maas-api-tls\") pod \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\" (UID: \"6d857bf2-535e-4af3-8822-32b6cb8b19bb\") "
Sep 17 23:23:04.243093 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.243052    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d857bf2-535e-4af3-8822-32b6cb8b19bb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d857bf2-535e-4af3-8822-32b6cb8b19bb" (UID: "6d857bf2-535e-4af3-8822-32b6cb8b19bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:23:04.245372 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.245341    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d857bf2-535e-4af3-8822-32b6cb8b19bb-kube-api-access-ndh2h" (OuterVolumeSpecName: "kube-api-access-ndh2h") pod "6d857bf2-535e-4af3-8822-32b6cb8b19bb" (UID: "6d857bf2-535e-4af3-8822-32b6cb8b19bb"). InnerVolumeSpecName "kube-api-access-ndh2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:23:04.247083 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.247050    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d857bf2-535e-4af3-8822-32b6cb8b19bb" (UID: "6d857bf2-535e-4af3-8822-32b6cb8b19bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:23:04.249159 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.249131    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6d857bf2-535e-4af3-8822-32b6cb8b19bb" (UID: "6d857bf2-535e-4af3-8822-32b6cb8b19bb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:23:04.276983 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.276957    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:04.302983 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.302959    2558 generic.go:356] "Generic (PLEG): container finished" podID="6d857bf2-535e-4af3-8822-32b6cb8b19bb" containerID="5e6773de839fa88895d351f2d7e4d99b5b01f59fedcffd195a29fb8020cc7acd" exitCode=0
Sep 17 23:23:04.303103 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.303006    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"
Sep 17 23:23:04.303103 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.303053    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj" event={"ID":"6d857bf2-535e-4af3-8822-32b6cb8b19bb","Type":"ContainerDied","Data":"5e6773de839fa88895d351f2d7e4d99b5b01f59fedcffd195a29fb8020cc7acd"}
Sep 17 23:23:04.303218 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.303112    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj" event={"ID":"6d857bf2-535e-4af3-8822-32b6cb8b19bb","Type":"ContainerDied","Data":"a4b214a545564dbf2b9c97c3426ed8814a41f606515f4b66a8fc954447ca8a6d"}
Sep 17 23:23:04.303218 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.303144    2558 scope.go:117] "RemoveContainer" containerID="5e6773de839fa88895d351f2d7e4d99b5b01f59fedcffd195a29fb8020cc7acd"
Sep 17 23:23:04.312222 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.312204    2558 scope.go:117] "RemoveContainer" containerID="5e6773de839fa88895d351f2d7e4d99b5b01f59fedcffd195a29fb8020cc7acd"
Sep 17 23:23:04.312470 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:23:04.312454    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e6773de839fa88895d351f2d7e4d99b5b01f59fedcffd195a29fb8020cc7acd\": container with ID starting with 5e6773de839fa88895d351f2d7e4d99b5b01f59fedcffd195a29fb8020cc7acd not found: ID does not exist" containerID="5e6773de839fa88895d351f2d7e4d99b5b01f59fedcffd195a29fb8020cc7acd"
Sep 17 23:23:04.312553 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.312479    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e6773de839fa88895d351f2d7e4d99b5b01f59fedcffd195a29fb8020cc7acd"} err="failed to get container status \"5e6773de839fa88895d351f2d7e4d99b5b01f59fedcffd195a29fb8020cc7acd\": rpc error: code = NotFound desc = could not find container \"5e6773de839fa88895d351f2d7e4d99b5b01f59fedcffd195a29fb8020cc7acd\": container with ID starting with 5e6773de839fa88895d351f2d7e4d99b5b01f59fedcffd195a29fb8020cc7acd not found: ID does not exist"
Sep 17 23:23:04.325354 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.325334    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"]
Sep 17 23:23:04.329504 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.329484    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d683cd6a-64cc849644-9tcxj"]
Sep 17 23:23:04.344471 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.344448    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndh2h\" (UniqueName: \"kubernetes.io/projected/6d857bf2-535e-4af3-8822-32b6cb8b19bb-kube-api-access-ndh2h\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:04.344471 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.344472    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:04.344618 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.344481    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d857bf2-535e-4af3-8822-32b6cb8b19bb-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:04.344618 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.344490    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d857bf2-535e-4af3-8822-32b6cb8b19bb-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:04.713139 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:04.713111    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:05.000975 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.000884    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"]
Sep 17 23:23:05.001440 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.001422    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51a49042-43e1-43de-a08f-6a7457d58bba" containerName="istio-proxy"
Sep 17 23:23:05.001498 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.001442    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="51a49042-43e1-43de-a08f-6a7457d58bba" containerName="istio-proxy"
Sep 17 23:23:05.001498 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.001451    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d857bf2-535e-4af3-8822-32b6cb8b19bb" containerName="maas-api"
Sep 17 23:23:05.001498 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.001456    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d857bf2-535e-4af3-8822-32b6cb8b19bb" containerName="maas-api"
Sep 17 23:23:05.001659 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.001537    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d857bf2-535e-4af3-8822-32b6cb8b19bb" containerName="maas-api"
Sep 17 23:23:05.004508 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.004485    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.006933 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.006904    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-cc9b429c-openshift-default-dockercfg-tqjbp\""
Sep 17 23:23:05.016411 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.016389    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"]
Sep 17 23:23:05.025701 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.025670    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:05.126354 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.126320    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:05.151705 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.151675    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-workload-certs\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.151876 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.151759    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-envoy\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.151876 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.151790    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istiod-ca-cert\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.151876 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.151819    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-data\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.151876 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.151854    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn2tf\" (UniqueName: \"kubernetes.io/projected/05781a65-0252-4a3d-bbaf-d86fe3e6285d-kube-api-access-tn2tf\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.152114 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.151973    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-credential-socket\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.152114 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.152068    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-workload-socket\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.152193 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.152129    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-podinfo\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.152193 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.152171    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-token\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.252605 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.252534    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-data\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.252605 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.252580    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn2tf\" (UniqueName: \"kubernetes.io/projected/05781a65-0252-4a3d-bbaf-d86fe3e6285d-kube-api-access-tn2tf\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.252760 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.252610    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-credential-socket\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.252760 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.252639    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-workload-socket\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.252900 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.252790    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-podinfo\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.252900 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.252842    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-token\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.252900 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.252890    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-workload-certs\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.253400 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.252894    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-data\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.253400 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.252929    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-workload-socket\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.253400 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.252965    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-credential-socket\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.253400 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.252983    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-envoy\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.253400 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.253019    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istiod-ca-cert\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.253400 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.253199    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-workload-certs\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.253749 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.253593    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istiod-ca-cert\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.255117 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.255093    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-envoy\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.255190 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.255134    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-podinfo\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.262931 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.262910    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-token\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.264344 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.264323    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn2tf\" (UniqueName: \"kubernetes.io/projected/05781a65-0252-4a3d-bbaf-d86fe3e6285d-kube-api-access-tn2tf\") pod \"e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") " pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.315879 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.315857    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:05.396851 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.396825    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:05.592797 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.592771    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"]
Sep 17 23:23:05.594422 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:23:05.594403    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05781a65_0252_4a3d_bbaf_d86fe3e6285d.slice/crio-5870ffb39788c5c73acf317b73ae47715f8cbe53e8607ed0a038547a36edce45 WatchSource:0}: Error finding container 5870ffb39788c5c73acf317b73ae47715f8cbe53e8607ed0a038547a36edce45: Status 404 returned error can't find the container with id 5870ffb39788c5c73acf317b73ae47715f8cbe53e8607ed0a038547a36edce45
Sep 17 23:23:05.596662 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.596634    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:23:05.596735 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.596695    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:23:05.596735 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.596724    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:23:05.932759 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:05.932688    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d857bf2-535e-4af3-8822-32b6cb8b19bb" path="/var/lib/kubelet/pods/6d857bf2-535e-4af3-8822-32b6cb8b19bb/volumes"
Sep 17 23:23:06.055408 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:06.055377    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:06.315920 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:06.315886    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd" event={"ID":"05781a65-0252-4a3d-bbaf-d86fe3e6285d","Type":"ContainerStarted","Data":"12defce1545003085484b71f2620d1a2bee7096b649d03c82d00e45c3b62ca91"}
Sep 17 23:23:06.316087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:06.315943    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd" event={"ID":"05781a65-0252-4a3d-bbaf-d86fe3e6285d","Type":"ContainerStarted","Data":"5870ffb39788c5c73acf317b73ae47715f8cbe53e8607ed0a038547a36edce45"}
Sep 17 23:23:06.834078 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:06.834043    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:07.316930 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:07.316894    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:07.323003 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:07.322977    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:07.343305 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:07.343255    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd" podStartSLOduration=3.3432372040000002 podStartE2EDuration="3.343237204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:23:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:23:06.341300322 +0000 UTC m=+1010.521938761" watchObservedRunningTime="2026-09-17 23:23:07.343237204 +0000 UTC m=+1011.523875644"
Sep 17 23:23:07.610869 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:07.610787    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:07.721559 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:07.721470    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:07.817943 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:07.817893    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:08.326875 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:08.326840    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:08.327637 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:08.327618    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:08.589404 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:08.589330    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:09.273538 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.273487    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"]
Sep 17 23:23:09.277218 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.277196    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:09.279599 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.279578    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-cc9b429c-serving-cert\""
Sep 17 23:23:09.290128 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.290105    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"]
Sep 17 23:23:09.313737 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.313712    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:09.390637 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.390610    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-metrics-certs\") pod \"maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:09.391116 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.391095    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-maas-api-tls\") pod \"maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:09.391270 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.391254    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-openshift-service-ca\") pod \"maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:09.391412 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.391397    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqvz6\" (UniqueName: \"kubernetes.io/projected/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-kube-api-access-pqvz6\") pod \"maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:09.442938 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.442910    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:09.492148 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.492121    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-metrics-certs\") pod \"maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:09.492261 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.492174    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-maas-api-tls\") pod \"maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:09.492261 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.492213    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-openshift-service-ca\") pod \"maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:09.492261 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.492238    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqvz6\" (UniqueName: \"kubernetes.io/projected/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-kube-api-access-pqvz6\") pod \"maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:09.492430 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:23:09.492267    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-serving-cert: secret "maas-api-e2e-ait-cc9b429c-serving-cert" not found
Sep 17 23:23:09.492430 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:23:09.492327    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-maas-api-tls podName:ed63a2d7-d32d-4a26-a857-816ffa0a8b70 nodeName:}" failed. No retries permitted until 2026-09-17 23:23:09.992308603 +0000 UTC m=+1014.172947038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-maas-api-tls") pod "maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2" (UID: "ed63a2d7-d32d-4a26-a857-816ffa0a8b70") : secret "maas-api-e2e-ait-cc9b429c-serving-cert" not found
Sep 17 23:23:09.492956 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.492937    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-openshift-service-ca\") pod \"maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:09.494549 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.494529    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-metrics-certs\") pod \"maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:09.500941 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.500923    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqvz6\" (UniqueName: \"kubernetes.io/projected/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-kube-api-access-pqvz6\") pod \"maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:09.897531 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.897480    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:09.990237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.990212    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:09.997461 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.997438    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-maas-api-tls\") pod \"maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:09.999810 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:09.999779    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-maas-api-tls\") pod \"maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:10.126287 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:10.126257    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:10.189903 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:10.189882    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:10.458540 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:10.458441    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:10.466767 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:10.466744    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"]
Sep 17 23:23:10.468479 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:23:10.468458    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded63a2d7_d32d_4a26_a857_816ffa0a8b70.slice/crio-43ed4bf8787f4a1229e069c74997e080505d814fb5951c6dafe4d69670bebb28 WatchSource:0}: Error finding container 43ed4bf8787f4a1229e069c74997e080505d814fb5951c6dafe4d69670bebb28: Status 404 returned error can't find the container with id 43ed4bf8787f4a1229e069c74997e080505d814fb5951c6dafe4d69670bebb28
Sep 17 23:23:11.343293 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:11.343252    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2" event={"ID":"ed63a2d7-d32d-4a26-a857-816ffa0a8b70","Type":"ContainerStarted","Data":"e743444dc060d2f02c0785a651b5babccab4531cf5e3e12eae439487c2725e1d"}
Sep 17 23:23:11.343450 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:11.343312    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:11.343450 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:11.343332    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2" event={"ID":"ed63a2d7-d32d-4a26-a857-816ffa0a8b70","Type":"ContainerStarted","Data":"43ed4bf8787f4a1229e069c74997e080505d814fb5951c6dafe4d69670bebb28"}
Sep 17 23:23:11.364195 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:11.364137    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2" podStartSLOduration=2.003968496 podStartE2EDuration="2.36411894s" totalImagesPullingTime="360.150444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:23:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:23:11.360186274 +0000 UTC m=+1015.540824716" watchObservedRunningTime="2026-09-17 23:23:11.36411894 +0000 UTC m=+1015.544757379"
Sep 17 23:23:11.460916 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:11.460888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:12.144743 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:12.144710    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:12.846960 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:12.846929    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:13.547575 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:13.547506    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:17.353911 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:17.353880    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:29.439900 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:29.439867    2558 generic.go:356] "Generic (PLEG): container finished" podID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" containerID="12defce1545003085484b71f2620d1a2bee7096b649d03c82d00e45c3b62ca91" exitCode=0
Sep 17 23:23:29.440294 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:29.439919    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd" event={"ID":"05781a65-0252-4a3d-bbaf-d86fe3e6285d","Type":"ContainerDied","Data":"12defce1545003085484b71f2620d1a2bee7096b649d03c82d00e45c3b62ca91"}
Sep 17 23:23:29.440294 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:29.440232    2558 scope.go:117] "RemoveContainer" containerID="12defce1545003085484b71f2620d1a2bee7096b649d03c82d00e45c3b62ca91"
Sep 17 23:23:29.441357 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:29.441330    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:23:29.441413 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:29.441395    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:23:29.441448 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:29.441425    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:23:30.449037 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:30.448986    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd" event={"ID":"05781a65-0252-4a3d-bbaf-d86fe3e6285d","Type":"ContainerStarted","Data":"7814b4f9a3fa74e971f73078a816f813aecad53a2f0e51b7dc3fb01b8005dde1"}
Sep 17 23:23:31.316457 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:31.316423    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:31.316677 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:31.316575    2558 patch_prober.go:28] interesting pod/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 17 23:23:31.316677 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:31.316639    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd" podUID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 17 23:23:31.766464 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:31.766300    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:32.317086 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:32.317049    2558 patch_prober.go:28] interesting pod/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 17 23:23:32.317267 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:32.317115    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd" podUID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 17 23:23:32.477527 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:32.477484    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:33.181779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:33.181750    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:33.316825 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:33.316795    2558 patch_prober.go:28] interesting pod/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 17 23:23:33.316982 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:33.316850    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd" podUID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 17 23:23:34.003955 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:34.003923    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:34.317124 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:34.317038    2558 patch_prober.go:28] interesting pod/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 17 23:23:34.317124 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:34.317102    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd" podUID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 17 23:23:34.871694 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:34.871653    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:35.501717 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:35.501682    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:35.502285 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:35.502261    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:35.503253 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:35.503228    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:35.593780 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:35.593744    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:36.370100 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:36.370070    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:37.095620 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:37.095591    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:37.851770 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:37.851744    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:38.030774 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:38.030747    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:38.573462 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:38.573436    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:39.241376 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.241345    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"]
Sep 17 23:23:39.241595 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.241575    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2" podUID="ed63a2d7-d32d-4a26-a857-816ffa0a8b70" containerName="maas-api" containerID="cri-o://e743444dc060d2f02c0785a651b5babccab4531cf5e3e12eae439487c2725e1d" gracePeriod=30
Sep 17 23:23:39.273149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.273101    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:39.351174 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.351142    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:39.451381 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.451351    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:39.479652 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.479629    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:39.507873 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.507800    2558 generic.go:356] "Generic (PLEG): container finished" podID="ed63a2d7-d32d-4a26-a857-816ffa0a8b70" containerID="e743444dc060d2f02c0785a651b5babccab4531cf5e3e12eae439487c2725e1d" exitCode=0
Sep 17 23:23:39.507873 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.507837    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2" event={"ID":"ed63a2d7-d32d-4a26-a857-816ffa0a8b70","Type":"ContainerDied","Data":"e743444dc060d2f02c0785a651b5babccab4531cf5e3e12eae439487c2725e1d"}
Sep 17 23:23:39.507873 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.507851    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"
Sep 17 23:23:39.508155 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.507898    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2" event={"ID":"ed63a2d7-d32d-4a26-a857-816ffa0a8b70","Type":"ContainerDied","Data":"43ed4bf8787f4a1229e069c74997e080505d814fb5951c6dafe4d69670bebb28"}
Sep 17 23:23:39.508155 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.507919    2558 scope.go:117] "RemoveContainer" containerID="e743444dc060d2f02c0785a651b5babccab4531cf5e3e12eae439487c2725e1d"
Sep 17 23:23:39.518017 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.518000    2558 scope.go:117] "RemoveContainer" containerID="e743444dc060d2f02c0785a651b5babccab4531cf5e3e12eae439487c2725e1d"
Sep 17 23:23:39.518289 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:23:39.518269    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e743444dc060d2f02c0785a651b5babccab4531cf5e3e12eae439487c2725e1d\": container with ID starting with e743444dc060d2f02c0785a651b5babccab4531cf5e3e12eae439487c2725e1d not found: ID does not exist" containerID="e743444dc060d2f02c0785a651b5babccab4531cf5e3e12eae439487c2725e1d"
Sep 17 23:23:39.518364 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.518296    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e743444dc060d2f02c0785a651b5babccab4531cf5e3e12eae439487c2725e1d"} err="failed to get container status \"e743444dc060d2f02c0785a651b5babccab4531cf5e3e12eae439487c2725e1d\": rpc error: code = NotFound desc = could not find container \"e743444dc060d2f02c0785a651b5babccab4531cf5e3e12eae439487c2725e1d\": container with ID starting with e743444dc060d2f02c0785a651b5babccab4531cf5e3e12eae439487c2725e1d not found: ID does not exist"
Sep 17 23:23:39.548328 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.548307    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqvz6\" (UniqueName: \"kubernetes.io/projected/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-kube-api-access-pqvz6\") pod \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") "
Sep 17 23:23:39.548415 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.548350    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-metrics-certs\") pod \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") "
Sep 17 23:23:39.548459 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.548415    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-maas-api-tls\") pod \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") "
Sep 17 23:23:39.548599 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.548577    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-openshift-service-ca\") pod \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\" (UID: \"ed63a2d7-d32d-4a26-a857-816ffa0a8b70\") "
Sep 17 23:23:39.548882 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.548858    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ed63a2d7-d32d-4a26-a857-816ffa0a8b70" (UID: "ed63a2d7-d32d-4a26-a857-816ffa0a8b70"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:23:39.550302 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.550275    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ed63a2d7-d32d-4a26-a857-816ffa0a8b70" (UID: "ed63a2d7-d32d-4a26-a857-816ffa0a8b70"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:23:39.550715 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.550694    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ed63a2d7-d32d-4a26-a857-816ffa0a8b70" (UID: "ed63a2d7-d32d-4a26-a857-816ffa0a8b70"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:23:39.550828 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.550809    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-kube-api-access-pqvz6" (OuterVolumeSpecName: "kube-api-access-pqvz6") pod "ed63a2d7-d32d-4a26-a857-816ffa0a8b70" (UID: "ed63a2d7-d32d-4a26-a857-816ffa0a8b70"). InnerVolumeSpecName "kube-api-access-pqvz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:23:39.550894 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.550819    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:39.649997 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.649962    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:39.649997 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.649995    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:39.650396 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.650009    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqvz6\" (UniqueName: \"kubernetes.io/projected/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-kube-api-access-pqvz6\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:39.650396 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.650017    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed63a2d7-d32d-4a26-a857-816ffa0a8b70-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:39.830193 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.830168    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"]
Sep 17 23:23:39.835340 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.835318    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cc9b429c-5bcf7b5867-rm5v2"]
Sep 17 23:23:39.932918 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.932892    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed63a2d7-d32d-4a26-a857-816ffa0a8b70" path="/var/lib/kubelet/pods/ed63a2d7-d32d-4a26-a857-816ffa0a8b70/volumes"
Sep 17 23:23:39.937677 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:39.937657    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:40.638029 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:40.637991    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"]
Sep 17 23:23:40.638270 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:40.638237    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd" podUID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" containerName="istio-proxy" containerID="cri-o://7814b4f9a3fa74e971f73078a816f813aecad53a2f0e51b7dc3fb01b8005dde1" gracePeriod=30
Sep 17 23:23:40.650385 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:40.650360    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:40.753966 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:40.753938    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:41.080579 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:41.080544    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:41.454709 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:41.454659    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:41.681605 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:41.681574    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:41.826541 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:41.826461    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:42.505460 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:42.505427    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:43.317692 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:43.317656    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:43.948015 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:43.947984    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:44.047295 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:44.047264    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:44.181233 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:44.181204    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:45.098273 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:45.098225    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:45.890265 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:45.890242    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:46.002081 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002057    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-data\") pod \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") "
Sep 17 23:23:46.002199 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002098    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-workload-socket\") pod \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") "
Sep 17 23:23:46.002199 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002151    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-envoy\") pod \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") "
Sep 17 23:23:46.002199 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002177    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-token\") pod \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") "
Sep 17 23:23:46.002345 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002243    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istiod-ca-cert\") pod \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") "
Sep 17 23:23:46.002345 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002267    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-credential-socket\") pod \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") "
Sep 17 23:23:46.002345 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002291    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-podinfo\") pod \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") "
Sep 17 23:23:46.002345 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002328    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-data" (OuterVolumeSpecName: "istio-data") pod "05781a65-0252-4a3d-bbaf-d86fe3e6285d" (UID: "05781a65-0252-4a3d-bbaf-d86fe3e6285d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:23:46.002567 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002358    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-workload-certs\") pod \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") "
Sep 17 23:23:46.002567 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002383    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tn2tf\" (UniqueName: \"kubernetes.io/projected/05781a65-0252-4a3d-bbaf-d86fe3e6285d-kube-api-access-tn2tf\") pod \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\" (UID: \"05781a65-0252-4a3d-bbaf-d86fe3e6285d\") "
Sep 17 23:23:46.002567 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002381    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "05781a65-0252-4a3d-bbaf-d86fe3e6285d" (UID: "05781a65-0252-4a3d-bbaf-d86fe3e6285d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:23:46.002748 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002601    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "05781a65-0252-4a3d-bbaf-d86fe3e6285d" (UID: "05781a65-0252-4a3d-bbaf-d86fe3e6285d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:23:46.002748 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002643    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "05781a65-0252-4a3d-bbaf-d86fe3e6285d" (UID: "05781a65-0252-4a3d-bbaf-d86fe3e6285d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:23:46.002748 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002684    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "05781a65-0252-4a3d-bbaf-d86fe3e6285d" (UID: "05781a65-0252-4a3d-bbaf-d86fe3e6285d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:23:46.002904 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002817    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-data\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:46.002904 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002836    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-workload-socket\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:46.002904 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002850    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istiod-ca-cert\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:46.002904 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002863    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-credential-socket\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:46.002904 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.002875    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-workload-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:46.004457 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.004434    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "05781a65-0252-4a3d-bbaf-d86fe3e6285d" (UID: "05781a65-0252-4a3d-bbaf-d86fe3e6285d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:23:46.004457 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.004448    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-token" (OuterVolumeSpecName: "istio-token") pod "05781a65-0252-4a3d-bbaf-d86fe3e6285d" (UID: "05781a65-0252-4a3d-bbaf-d86fe3e6285d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:23:46.004611 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.004482    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "05781a65-0252-4a3d-bbaf-d86fe3e6285d" (UID: "05781a65-0252-4a3d-bbaf-d86fe3e6285d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:23:46.004611 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.004546    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05781a65-0252-4a3d-bbaf-d86fe3e6285d-kube-api-access-tn2tf" (OuterVolumeSpecName: "kube-api-access-tn2tf") pod "05781a65-0252-4a3d-bbaf-d86fe3e6285d" (UID: "05781a65-0252-4a3d-bbaf-d86fe3e6285d"). InnerVolumeSpecName "kube-api-access-tn2tf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:23:46.104075 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.104048    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-envoy\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:46.104075 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.104071    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-token\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:46.104075 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.104080    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05781a65-0252-4a3d-bbaf-d86fe3e6285d-istio-podinfo\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:46.104423 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.104088    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tn2tf\" (UniqueName: \"kubernetes.io/projected/05781a65-0252-4a3d-bbaf-d86fe3e6285d-kube-api-access-tn2tf\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:23:46.275371 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.275314    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:46.550725 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.550648    2558 generic.go:356] "Generic (PLEG): container finished" podID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" containerID="7814b4f9a3fa74e971f73078a816f813aecad53a2f0e51b7dc3fb01b8005dde1" exitCode=0
Sep 17 23:23:46.550725 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.550697    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"
Sep 17 23:23:46.550905 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.550729    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd" event={"ID":"05781a65-0252-4a3d-bbaf-d86fe3e6285d","Type":"ContainerDied","Data":"7814b4f9a3fa74e971f73078a816f813aecad53a2f0e51b7dc3fb01b8005dde1"}
Sep 17 23:23:46.550905 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.550788    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd" event={"ID":"05781a65-0252-4a3d-bbaf-d86fe3e6285d","Type":"ContainerDied","Data":"5870ffb39788c5c73acf317b73ae47715f8cbe53e8607ed0a038547a36edce45"}
Sep 17 23:23:46.550905 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.550810    2558 scope.go:117] "RemoveContainer" containerID="7814b4f9a3fa74e971f73078a816f813aecad53a2f0e51b7dc3fb01b8005dde1"
Sep 17 23:23:46.560499 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.560479    2558 scope.go:117] "RemoveContainer" containerID="12defce1545003085484b71f2620d1a2bee7096b649d03c82d00e45c3b62ca91"
Sep 17 23:23:46.568206 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.568190    2558 scope.go:117] "RemoveContainer" containerID="7814b4f9a3fa74e971f73078a816f813aecad53a2f0e51b7dc3fb01b8005dde1"
Sep 17 23:23:46.568466 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:23:46.568447    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7814b4f9a3fa74e971f73078a816f813aecad53a2f0e51b7dc3fb01b8005dde1\": container with ID starting with 7814b4f9a3fa74e971f73078a816f813aecad53a2f0e51b7dc3fb01b8005dde1 not found: ID does not exist" containerID="7814b4f9a3fa74e971f73078a816f813aecad53a2f0e51b7dc3fb01b8005dde1"
Sep 17 23:23:46.568466 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.568474    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7814b4f9a3fa74e971f73078a816f813aecad53a2f0e51b7dc3fb01b8005dde1"} err="failed to get container status \"7814b4f9a3fa74e971f73078a816f813aecad53a2f0e51b7dc3fb01b8005dde1\": rpc error: code = NotFound desc = could not find container \"7814b4f9a3fa74e971f73078a816f813aecad53a2f0e51b7dc3fb01b8005dde1\": container with ID starting with 7814b4f9a3fa74e971f73078a816f813aecad53a2f0e51b7dc3fb01b8005dde1 not found: ID does not exist"
Sep 17 23:23:46.568466 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.568490    2558 scope.go:117] "RemoveContainer" containerID="12defce1545003085484b71f2620d1a2bee7096b649d03c82d00e45c3b62ca91"
Sep 17 23:23:46.568885 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:23:46.568861    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12defce1545003085484b71f2620d1a2bee7096b649d03c82d00e45c3b62ca91\": container with ID starting with 12defce1545003085484b71f2620d1a2bee7096b649d03c82d00e45c3b62ca91 not found: ID does not exist" containerID="12defce1545003085484b71f2620d1a2bee7096b649d03c82d00e45c3b62ca91"
Sep 17 23:23:46.568961 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.568893    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12defce1545003085484b71f2620d1a2bee7096b649d03c82d00e45c3b62ca91"} err="failed to get container status \"12defce1545003085484b71f2620d1a2bee7096b649d03c82d00e45c3b62ca91\": rpc error: code = NotFound desc = could not find container \"12defce1545003085484b71f2620d1a2bee7096b649d03c82d00e45c3b62ca91\": container with ID starting with 12defce1545003085484b71f2620d1a2bee7096b649d03c82d00e45c3b62ca91 not found: ID does not exist"
Sep 17 23:23:46.572082 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.572061    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"]
Sep 17 23:23:46.574902 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.574879    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-cc9b429c-openshift-default-698bc6cd6-g6kcd"]
Sep 17 23:23:46.991218 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:46.991181    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:47.756341 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:47.756308    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:47.933121 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:47.933090    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" path="/var/lib/kubelet/pods/05781a65-0252-4a3d-bbaf-d86fe3e6285d/volumes"
Sep 17 23:23:48.531659 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.531629    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"]
Sep 17 23:23:48.532128 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.532114    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" containerName="istio-proxy"
Sep 17 23:23:48.532178 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.532132    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" containerName="istio-proxy"
Sep 17 23:23:48.532178 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.532154    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed63a2d7-d32d-4a26-a857-816ffa0a8b70" containerName="maas-api"
Sep 17 23:23:48.532178 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.532162    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed63a2d7-d32d-4a26-a857-816ffa0a8b70" containerName="maas-api"
Sep 17 23:23:48.532281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.532258    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed63a2d7-d32d-4a26-a857-816ffa0a8b70" containerName="maas-api"
Sep 17 23:23:48.532281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.532273    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" containerName="istio-proxy"
Sep 17 23:23:48.532338 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.532283    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" containerName="istio-proxy"
Sep 17 23:23:48.537240 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.537218    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:48.539938 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.539915    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-4f19d32d-serving-cert\""
Sep 17 23:23:48.543188 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.543166    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"]
Sep 17 23:23:48.563440 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.563417    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:48.625638 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.625615    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:48.626773 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.626751    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bk7b\" (UniqueName: \"kubernetes.io/projected/8c765899-1725-4392-a11c-9098a5f4e452-kube-api-access-9bk7b\") pod \"maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:48.626869 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.626796    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-metrics-certs\") pod \"maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:48.626869 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.626862    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-maas-api-tls\") pod \"maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:48.626969 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.626940    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c765899-1725-4392-a11c-9098a5f4e452-openshift-service-ca\") pod \"maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:48.728390 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.728364    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-maas-api-tls\") pod \"maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:48.728575 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.728414    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c765899-1725-4392-a11c-9098a5f4e452-openshift-service-ca\") pod \"maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:48.728575 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.728470    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bk7b\" (UniqueName: \"kubernetes.io/projected/8c765899-1725-4392-a11c-9098a5f4e452-kube-api-access-9bk7b\") pod \"maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:48.728575 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.728492    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-metrics-certs\") pod \"maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:48.728575 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:23:48.728553    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-serving-cert: secret "maas-api-e2e-derive-4f19d32d-serving-cert" not found
Sep 17 23:23:48.728847 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:23:48.728625    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-maas-api-tls podName:8c765899-1725-4392-a11c-9098a5f4e452 nodeName:}" failed. No retries permitted until 2026-09-17 23:23:49.228603479 +0000 UTC m=+1053.409241898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-maas-api-tls") pod "maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c" (UID: "8c765899-1725-4392-a11c-9098a5f4e452") : secret "maas-api-e2e-derive-4f19d32d-serving-cert" not found
Sep 17 23:23:48.729129 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.729104    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c765899-1725-4392-a11c-9098a5f4e452-openshift-service-ca\") pod \"maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:48.731008 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.730981    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-metrics-certs\") pod \"maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:48.735992 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.735971    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bk7b\" (UniqueName: \"kubernetes.io/projected/8c765899-1725-4392-a11c-9098a5f4e452-kube-api-access-9bk7b\") pod \"maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:48.823482 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:48.823423    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:49.118510 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:49.118433    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:49.216346 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:49.216319    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:23:49.232633 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:49.232605    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-maas-api-tls\") pod \"maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:49.235414 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:49.235389    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-maas-api-tls\") pod \"maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:49.451072 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:49.451041    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:49.719019 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:49.718995    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"]
Sep 17 23:23:49.720039 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:23:49.720014    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c765899_1725_4392_a11c_9098a5f4e452.slice/crio-875199f792fcf9f9b3674be2afa59abcf403f645f0a7a65c17ba987f6f545f62 WatchSource:0}: Error finding container 875199f792fcf9f9b3674be2afa59abcf403f645f0a7a65c17ba987f6f545f62: Status 404 returned error can't find the container with id 875199f792fcf9f9b3674be2afa59abcf403f645f0a7a65c17ba987f6f545f62
Sep 17 23:23:49.721269 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:49.721253    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:23:50.574475 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:50.574395    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c" event={"ID":"8c765899-1725-4392-a11c-9098a5f4e452","Type":"ContainerStarted","Data":"91104f4e40b99ccf0a12b53971b129c4404b7d04551d192083c36b8a09844741"}
Sep 17 23:23:50.574475 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:50.574444    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c" event={"ID":"8c765899-1725-4392-a11c-9098a5f4e452","Type":"ContainerStarted","Data":"875199f792fcf9f9b3674be2afa59abcf403f645f0a7a65c17ba987f6f545f62"}
Sep 17 23:23:50.574884 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:50.574533    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:23:50.591781 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:50.591738    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c" podStartSLOduration=1.989700813 podStartE2EDuration="2.591723357s" totalImagesPullingTime="602.022544ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:23:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:23:50.589412414 +0000 UTC m=+1054.770050854" watchObservedRunningTime="2026-09-17 23:23:50.591723357 +0000 UTC m=+1054.772361795"
Sep 17 23:23:56.585072 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:23:56.585044    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:24:03.981562 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:03.981413    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:09.547491 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:09.547459    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:11.058980 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:11.058935    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:11.843661 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:11.841202    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:12.134352 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.134281    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"]
Sep 17 23:24:12.134706 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.134672    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" containerName="istio-proxy"
Sep 17 23:24:12.134706 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.134684    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="05781a65-0252-4a3d-bbaf-d86fe3e6285d" containerName="istio-proxy"
Sep 17 23:24:12.137691 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.137674    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.140889 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.140870    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-983490\"/\"default-dockercfg-k5shj\""
Sep 17 23:24:12.140889 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.140882    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-983490\"/\"simulator-w4-983490-kserve-self-signed-certs\""
Sep 17 23:24:12.141044 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.140898    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-983490\"/\"kube-root-ca.crt\""
Sep 17 23:24:12.141044 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.140933    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-983490\"/\"openshift-service-ca.crt\""
Sep 17 23:24:12.147771 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.147749    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"]
Sep 17 23:24:12.219254 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.219228    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-tmp-dir\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.219388 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.219263    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b14167e9-c982-44ef-832c-61d06d82f44b-tls-certs\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.219388 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.219290    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-model-cache\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.219388 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.219322    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpkvr\" (UniqueName: \"kubernetes.io/projected/b14167e9-c982-44ef-832c-61d06d82f44b-kube-api-access-hpkvr\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.219501 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.219410    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-home\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.219501 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.219466    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-dshm\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.241158 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.241130    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:12.319899 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.319871    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-tmp-dir\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.320016 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.319907    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b14167e9-c982-44ef-832c-61d06d82f44b-tls-certs\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.320016 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.319940    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-model-cache\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.320016 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.319975    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpkvr\" (UniqueName: \"kubernetes.io/projected/b14167e9-c982-44ef-832c-61d06d82f44b-kube-api-access-hpkvr\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.320016 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.320009    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-home\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.320238 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.320055    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-dshm\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.320301 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.320250    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-tmp-dir\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.320367 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.320313    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-model-cache\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.320367 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.320352    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-home\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.322149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.322129    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-dshm\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.322332 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.322314    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b14167e9-c982-44ef-832c-61d06d82f44b-tls-certs\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.327399 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.327378    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpkvr\" (UniqueName: \"kubernetes.io/projected/b14167e9-c982-44ef-832c-61d06d82f44b-kube-api-access-hpkvr\") pod \"simulator-w4-983490-kserve-cf97f97b4-2sbzp\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") " pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.447066 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.447039    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:12.673389 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.673357    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:12.718195 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:12.718173    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"]
Sep 17 23:24:12.720129 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:24:12.720102    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb14167e9_c982_44ef_832c_61d06d82f44b.slice/crio-8b1b159deac3596bfb1d9a68d7d4560b72410bfb0c3e0bca7757118551125d7d WatchSource:0}: Error finding container 8b1b159deac3596bfb1d9a68d7d4560b72410bfb0c3e0bca7757118551125d7d: Status 404 returned error can't find the container with id 8b1b159deac3596bfb1d9a68d7d4560b72410bfb0c3e0bca7757118551125d7d
Sep 17 23:24:13.392841 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:13.392810    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:13.695548 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:13.695503    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp" event={"ID":"b14167e9-c982-44ef-832c-61d06d82f44b","Type":"ContainerStarted","Data":"ae0438ab7c4cf1fa047b636260b19f177659963e6d84de3f8ef9fd2f6c878745"}
Sep 17 23:24:13.695726 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:13.695561    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp" event={"ID":"b14167e9-c982-44ef-832c-61d06d82f44b","Type":"ContainerStarted","Data":"8b1b159deac3596bfb1d9a68d7d4560b72410bfb0c3e0bca7757118551125d7d"}
Sep 17 23:24:13.712467 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:13.712417    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp" podStartSLOduration=1.712399365 podStartE2EDuration="1.712399365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:24:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:24:13.711903849 +0000 UTC m=+1077.892542287" watchObservedRunningTime="2026-09-17 23:24:13.712399365 +0000 UTC m=+1077.893037805"
Sep 17 23:24:14.357860 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:14.357829    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:15.069814 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:15.069780    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:15.793424 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:15.793391    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:16.660444 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:16.660410    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:17.452333 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:17.452280    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:18.172355 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:18.172279    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:18.963923 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:18.963890    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:20.451197 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:20.451148    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:22.447448 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:22.447384    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:22.447855 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:22.447461    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:22.459336 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:22.459318    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:22.756434 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:22.756346    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:24:25.268209 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:25.268160    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:25.843394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:25.843363    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"]
Sep 17 23:24:25.843602 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:25.843578    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c" podUID="8c765899-1725-4392-a11c-9098a5f4e452" containerName="maas-api" containerID="cri-o://91104f4e40b99ccf0a12b53971b129c4404b7d04551d192083c36b8a09844741" gracePeriod=30
Sep 17 23:24:25.873588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:25.873560    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:25.947311 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:25.947274    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:26.013826 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.013790    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:26.091766 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.091737    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:24:26.112243 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.112044    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:26.119178 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.119155    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-metrics-certs\") pod \"8c765899-1725-4392-a11c-9098a5f4e452\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") "
Sep 17 23:24:26.119283 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.119216    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bk7b\" (UniqueName: \"kubernetes.io/projected/8c765899-1725-4392-a11c-9098a5f4e452-kube-api-access-9bk7b\") pod \"8c765899-1725-4392-a11c-9098a5f4e452\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") "
Sep 17 23:24:26.119283 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.119234    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c765899-1725-4392-a11c-9098a5f4e452-openshift-service-ca\") pod \"8c765899-1725-4392-a11c-9098a5f4e452\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") "
Sep 17 23:24:26.119283 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.119265    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-maas-api-tls\") pod \"8c765899-1725-4392-a11c-9098a5f4e452\" (UID: \"8c765899-1725-4392-a11c-9098a5f4e452\") "
Sep 17 23:24:26.119735 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.119706    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c765899-1725-4392-a11c-9098a5f4e452-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8c765899-1725-4392-a11c-9098a5f4e452" (UID: "8c765899-1725-4392-a11c-9098a5f4e452"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:24:26.121393 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.121363    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8c765899-1725-4392-a11c-9098a5f4e452" (UID: "8c765899-1725-4392-a11c-9098a5f4e452"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:24:26.121393 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.121362    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c765899-1725-4392-a11c-9098a5f4e452-kube-api-access-9bk7b" (OuterVolumeSpecName: "kube-api-access-9bk7b") pod "8c765899-1725-4392-a11c-9098a5f4e452" (UID: "8c765899-1725-4392-a11c-9098a5f4e452"). InnerVolumeSpecName "kube-api-access-9bk7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:24:26.121546 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.121403    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8c765899-1725-4392-a11c-9098a5f4e452" (UID: "8c765899-1725-4392-a11c-9098a5f4e452"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:24:26.175370 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.175350    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:26.221096 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.221017    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:24:26.221246 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.221101    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bk7b\" (UniqueName: \"kubernetes.io/projected/8c765899-1725-4392-a11c-9098a5f4e452-kube-api-access-9bk7b\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:24:26.221246 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.221118    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c765899-1725-4392-a11c-9098a5f4e452-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:24:26.221246 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.221140    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c765899-1725-4392-a11c-9098a5f4e452-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:24:26.279940 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.279911    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:26.490801 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.490774    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:26.767025 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.766943    2558 generic.go:356] "Generic (PLEG): container finished" podID="8c765899-1725-4392-a11c-9098a5f4e452" containerID="91104f4e40b99ccf0a12b53971b129c4404b7d04551d192083c36b8a09844741" exitCode=0
Sep 17 23:24:26.767025 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.766993    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"
Sep 17 23:24:26.767025 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.767002    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c" event={"ID":"8c765899-1725-4392-a11c-9098a5f4e452","Type":"ContainerDied","Data":"91104f4e40b99ccf0a12b53971b129c4404b7d04551d192083c36b8a09844741"}
Sep 17 23:24:26.767273 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.767042    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c" event={"ID":"8c765899-1725-4392-a11c-9098a5f4e452","Type":"ContainerDied","Data":"875199f792fcf9f9b3674be2afa59abcf403f645f0a7a65c17ba987f6f545f62"}
Sep 17 23:24:26.767273 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.767065    2558 scope.go:117] "RemoveContainer" containerID="91104f4e40b99ccf0a12b53971b129c4404b7d04551d192083c36b8a09844741"
Sep 17 23:24:26.776383 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.776345    2558 scope.go:117] "RemoveContainer" containerID="91104f4e40b99ccf0a12b53971b129c4404b7d04551d192083c36b8a09844741"
Sep 17 23:24:26.776716 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:24:26.776696    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91104f4e40b99ccf0a12b53971b129c4404b7d04551d192083c36b8a09844741\": container with ID starting with 91104f4e40b99ccf0a12b53971b129c4404b7d04551d192083c36b8a09844741 not found: ID does not exist" containerID="91104f4e40b99ccf0a12b53971b129c4404b7d04551d192083c36b8a09844741"
Sep 17 23:24:26.776801 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.776725    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91104f4e40b99ccf0a12b53971b129c4404b7d04551d192083c36b8a09844741"} err="failed to get container status \"91104f4e40b99ccf0a12b53971b129c4404b7d04551d192083c36b8a09844741\": rpc error: code = NotFound desc = could not find container \"91104f4e40b99ccf0a12b53971b129c4404b7d04551d192083c36b8a09844741\": container with ID starting with 91104f4e40b99ccf0a12b53971b129c4404b7d04551d192083c36b8a09844741 not found: ID does not exist"
Sep 17 23:24:26.789056 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.789032    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"]
Sep 17 23:24:26.793033 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.793009    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4f19d32d-6d48b796d8-scw7c"]
Sep 17 23:24:26.953061 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:26.953031    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:27.056329 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:27.056260    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:27.152246 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:27.152223    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:27.478707 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:27.478678    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:27.931364 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:27.931289    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c765899-1725-4392-a11c-9098a5f4e452" path="/var/lib/kubelet/pods/8c765899-1725-4392-a11c-9098a5f4e452/volumes"
Sep 17 23:24:28.212637 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:28.212611    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:28.535616 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:28.535507    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:29.280150 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:29.280120    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:29.994984 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:29.994952    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:30.696967 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:30.696935    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:33.758163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:33.758128    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:34.491332 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:34.491299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:34.603913 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:34.603858    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:34.690328 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:34.690296    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:34.792160 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:34.792094    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:35.193524 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:35.193496    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:47.260151 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:47.260119    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:51.479491 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:51.479457    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:52.218230 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:52.218197    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:52.921359 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:52.921314    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:53.765559 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:53.765486    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:54.468881 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:54.468850    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:55.162264 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:55.162231    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:55.970117 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:55.970082    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:58.564797 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:58.564763    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:58.664985 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:58.664957    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:58.854695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:58.854618    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:24:59.786358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:24:59.786324    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:00.488240 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:00.488209    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:01.126406 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:01.126370    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:01.372375 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:01.372343    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:02.178212 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:02.178181    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:03.178062 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:03.178032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:03.545576 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:03.545539    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"]
Sep 17 23:25:03.597154 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:03.597125    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:03.667804 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:03.667774    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:03.882640 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:03.882566    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:04.445667 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.445627    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"]
Sep 17 23:25:04.445880 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.445851    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk" podUID="de0c4bff-b9bd-4b7c-a1cc-ea50e863619d" containerName="maas-api" containerID="cri-o://c0dfc9ca755a0bbc3391cf117196cc0357a1fd288b5af69c3c765189268e6ddc" gracePeriod=30
Sep 17 23:25:04.476411 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.476373    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:04.577015 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.576981    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:04.696539 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.696465    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:04.697424 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.697409    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:25:04.804411 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.804375    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:04.846414 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.846390    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-metrics-certs\") pod \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") "
Sep 17 23:25:04.846556 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.846421    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-maas-api-tls\") pod \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") "
Sep 17 23:25:04.846556 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.846466    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tt8x6\" (UniqueName: \"kubernetes.io/projected/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-kube-api-access-tt8x6\") pod \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") "
Sep 17 23:25:04.846635 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.846604    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-openshift-service-ca\") pod \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\" (UID: \"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d\") "
Sep 17 23:25:04.846977 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.846948    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de0c4bff-b9bd-4b7c-a1cc-ea50e863619d" (UID: "de0c4bff-b9bd-4b7c-a1cc-ea50e863619d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:25:04.848600 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.848572    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-kube-api-access-tt8x6" (OuterVolumeSpecName: "kube-api-access-tt8x6") pod "de0c4bff-b9bd-4b7c-a1cc-ea50e863619d" (UID: "de0c4bff-b9bd-4b7c-a1cc-ea50e863619d"). InnerVolumeSpecName "kube-api-access-tt8x6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:25:04.848600 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.848585    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de0c4bff-b9bd-4b7c-a1cc-ea50e863619d" (UID: "de0c4bff-b9bd-4b7c-a1cc-ea50e863619d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:25:04.848732 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.848668    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "de0c4bff-b9bd-4b7c-a1cc-ea50e863619d" (UID: "de0c4bff-b9bd-4b7c-a1cc-ea50e863619d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:25:04.947820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.947757    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tt8x6\" (UniqueName: \"kubernetes.io/projected/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-kube-api-access-tt8x6\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:04.947820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.947782    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:04.947820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.947795    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:04.947820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.947807    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:04.965720 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.965697    2558 generic.go:356] "Generic (PLEG): container finished" podID="de0c4bff-b9bd-4b7c-a1cc-ea50e863619d" containerID="c0dfc9ca755a0bbc3391cf117196cc0357a1fd288b5af69c3c765189268e6ddc" exitCode=0
Sep 17 23:25:04.965813 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.965741    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"
Sep 17 23:25:04.965813 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.965786    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk" event={"ID":"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d","Type":"ContainerDied","Data":"c0dfc9ca755a0bbc3391cf117196cc0357a1fd288b5af69c3c765189268e6ddc"}
Sep 17 23:25:04.965886 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.965835    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk" event={"ID":"de0c4bff-b9bd-4b7c-a1cc-ea50e863619d","Type":"ContainerDied","Data":"bffb251837c41f6c31e37c893b09481c7f0674e38b6920c0e43249324fc6b723"}
Sep 17 23:25:04.965920 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.965862    2558 scope.go:117] "RemoveContainer" containerID="c0dfc9ca755a0bbc3391cf117196cc0357a1fd288b5af69c3c765189268e6ddc"
Sep 17 23:25:04.977068 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.977050    2558 scope.go:117] "RemoveContainer" containerID="c0dfc9ca755a0bbc3391cf117196cc0357a1fd288b5af69c3c765189268e6ddc"
Sep 17 23:25:04.977390 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:25:04.977367    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0dfc9ca755a0bbc3391cf117196cc0357a1fd288b5af69c3c765189268e6ddc\": container with ID starting with c0dfc9ca755a0bbc3391cf117196cc0357a1fd288b5af69c3c765189268e6ddc not found: ID does not exist" containerID="c0dfc9ca755a0bbc3391cf117196cc0357a1fd288b5af69c3c765189268e6ddc"
Sep 17 23:25:04.977447 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.977400    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0dfc9ca755a0bbc3391cf117196cc0357a1fd288b5af69c3c765189268e6ddc"} err="failed to get container status \"c0dfc9ca755a0bbc3391cf117196cc0357a1fd288b5af69c3c765189268e6ddc\": rpc error: code = NotFound desc = could not find container \"c0dfc9ca755a0bbc3391cf117196cc0357a1fd288b5af69c3c765189268e6ddc\": container with ID starting with c0dfc9ca755a0bbc3391cf117196cc0357a1fd288b5af69c3c765189268e6ddc not found: ID does not exist"
Sep 17 23:25:04.988347 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.988328    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"]
Sep 17 23:25:04.992385 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:04.992368    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-63b144-bb4b4d8f-zzsvk"]
Sep 17 23:25:05.014339 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:05.014318    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:05.187855 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:05.187822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:05.838373 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:05.838338    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:05.931453 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:05.931426    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de0c4bff-b9bd-4b7c-a1cc-ea50e863619d" path="/var/lib/kubelet/pods/de0c4bff-b9bd-4b7c-a1cc-ea50e863619d/volumes"
Sep 17 23:25:05.937540 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:05.937498    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:06.785352 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:06.785318    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:07.625772 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:07.625739    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:07.730601 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:07.730567    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:07.838191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:07.838165    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:07.933052 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:07.932989    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:08.489799 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:08.489772    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:09.090377 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.090342    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"]
Sep 17 23:25:09.090814 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.090569    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x" podUID="eb987c4b-f861-46c3-a2de-4c74a918f649" containerName="maas-api" containerID="cri-o://9ccad52791b182170be43fc710d36a3ac54c024293ce830403801b515dea81ef" gracePeriod=30
Sep 17 23:25:09.128187 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.128162    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:09.211394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.211368    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:09.309620 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.309586    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:09.332121 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.332103    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:25:09.409442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.409387    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:09.485070 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.485038    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb987c4b-f861-46c3-a2de-4c74a918f649-metrics-certs\") pod \"eb987c4b-f861-46c3-a2de-4c74a918f649\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") "
Sep 17 23:25:09.485223 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.485120    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcdd2\" (UniqueName: \"kubernetes.io/projected/eb987c4b-f861-46c3-a2de-4c74a918f649-kube-api-access-pcdd2\") pod \"eb987c4b-f861-46c3-a2de-4c74a918f649\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") "
Sep 17 23:25:09.485223 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.485165    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb987c4b-f861-46c3-a2de-4c74a918f649-maas-api-tls\") pod \"eb987c4b-f861-46c3-a2de-4c74a918f649\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") "
Sep 17 23:25:09.485306 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.485246    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb987c4b-f861-46c3-a2de-4c74a918f649-openshift-service-ca\") pod \"eb987c4b-f861-46c3-a2de-4c74a918f649\" (UID: \"eb987c4b-f861-46c3-a2de-4c74a918f649\") "
Sep 17 23:25:09.485698 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.485658    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb987c4b-f861-46c3-a2de-4c74a918f649-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eb987c4b-f861-46c3-a2de-4c74a918f649" (UID: "eb987c4b-f861-46c3-a2de-4c74a918f649"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:25:09.487446 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.487425    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb987c4b-f861-46c3-a2de-4c74a918f649-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eb987c4b-f861-46c3-a2de-4c74a918f649" (UID: "eb987c4b-f861-46c3-a2de-4c74a918f649"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:25:09.487785 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.487761    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb987c4b-f861-46c3-a2de-4c74a918f649-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eb987c4b-f861-46c3-a2de-4c74a918f649" (UID: "eb987c4b-f861-46c3-a2de-4c74a918f649"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:25:09.487785 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.487766    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb987c4b-f861-46c3-a2de-4c74a918f649-kube-api-access-pcdd2" (OuterVolumeSpecName: "kube-api-access-pcdd2") pod "eb987c4b-f861-46c3-a2de-4c74a918f649" (UID: "eb987c4b-f861-46c3-a2de-4c74a918f649"). InnerVolumeSpecName "kube-api-access-pcdd2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:25:09.586071 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.586045    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcdd2\" (UniqueName: \"kubernetes.io/projected/eb987c4b-f861-46c3-a2de-4c74a918f649-kube-api-access-pcdd2\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:09.586071 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.586068    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb987c4b-f861-46c3-a2de-4c74a918f649-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:09.586071 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.586077    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb987c4b-f861-46c3-a2de-4c74a918f649-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:09.586268 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.586085    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb987c4b-f861-46c3-a2de-4c74a918f649-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:09.705626 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.705595    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:09.808044 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.808016    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:09.993925 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.993851    2558 generic.go:356] "Generic (PLEG): container finished" podID="eb987c4b-f861-46c3-a2de-4c74a918f649" containerID="9ccad52791b182170be43fc710d36a3ac54c024293ce830403801b515dea81ef" exitCode=0
Sep 17 23:25:09.993925 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.993886    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"
Sep 17 23:25:09.993925 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.993901    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x" event={"ID":"eb987c4b-f861-46c3-a2de-4c74a918f649","Type":"ContainerDied","Data":"9ccad52791b182170be43fc710d36a3ac54c024293ce830403801b515dea81ef"}
Sep 17 23:25:09.994241 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.993956    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x" event={"ID":"eb987c4b-f861-46c3-a2de-4c74a918f649","Type":"ContainerDied","Data":"2fcc04fa8f983aec10b5f300306cf5ec86dd84a552882483bd4ddd43544f2c5d"}
Sep 17 23:25:09.994241 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:09.993992    2558 scope.go:117] "RemoveContainer" containerID="9ccad52791b182170be43fc710d36a3ac54c024293ce830403801b515dea81ef"
Sep 17 23:25:10.003193 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:10.003165    2558 scope.go:117] "RemoveContainer" containerID="9ccad52791b182170be43fc710d36a3ac54c024293ce830403801b515dea81ef"
Sep 17 23:25:10.003497 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:25:10.003478    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ccad52791b182170be43fc710d36a3ac54c024293ce830403801b515dea81ef\": container with ID starting with 9ccad52791b182170be43fc710d36a3ac54c024293ce830403801b515dea81ef not found: ID does not exist" containerID="9ccad52791b182170be43fc710d36a3ac54c024293ce830403801b515dea81ef"
Sep 17 23:25:10.003585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:10.003507    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ccad52791b182170be43fc710d36a3ac54c024293ce830403801b515dea81ef"} err="failed to get container status \"9ccad52791b182170be43fc710d36a3ac54c024293ce830403801b515dea81ef\": rpc error: code = NotFound desc = could not find container \"9ccad52791b182170be43fc710d36a3ac54c024293ce830403801b515dea81ef\": container with ID starting with 9ccad52791b182170be43fc710d36a3ac54c024293ce830403801b515dea81ef not found: ID does not exist"
Sep 17 23:25:10.012649 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:10.012627    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"]
Sep 17 23:25:10.020623 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:10.020602    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-983490-8b856bd79-6x76x"]
Sep 17 23:25:10.091972 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:10.091948    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:10.733148 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:10.733115    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:11.563720 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:11.563686    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:11.931802 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:11.931730    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb987c4b-f861-46c3-a2de-4c74a918f649" path="/var/lib/kubelet/pods/eb987c4b-f861-46c3-a2de-4c74a918f649/volumes"
Sep 17 23:25:12.263192 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:12.263160    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:15.564730 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:15.564698    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:15.635630 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:15.635599    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:15.830321 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:15.830231    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:18.564782 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.564745    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp" podUID="b14167e9-c982-44ef-832c-61d06d82f44b" containerName="main" containerID="cri-o://ae0438ab7c4cf1fa047b636260b19f177659963e6d84de3f8ef9fd2f6c878745" gracePeriod=45
Sep 17 23:25:18.812577 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.812552    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:25:18.850782 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.850713    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-dshm\") pod \"b14167e9-c982-44ef-832c-61d06d82f44b\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") "
Sep 17 23:25:18.850782 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.850757    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-model-cache\") pod \"b14167e9-c982-44ef-832c-61d06d82f44b\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") "
Sep 17 23:25:18.850999 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.850797    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-tmp-dir\") pod \"b14167e9-c982-44ef-832c-61d06d82f44b\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") "
Sep 17 23:25:18.850999 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.850836    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpkvr\" (UniqueName: \"kubernetes.io/projected/b14167e9-c982-44ef-832c-61d06d82f44b-kube-api-access-hpkvr\") pod \"b14167e9-c982-44ef-832c-61d06d82f44b\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") "
Sep 17 23:25:18.850999 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.850893    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b14167e9-c982-44ef-832c-61d06d82f44b-tls-certs\") pod \"b14167e9-c982-44ef-832c-61d06d82f44b\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") "
Sep 17 23:25:18.850999 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.850916    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-home\") pod \"b14167e9-c982-44ef-832c-61d06d82f44b\" (UID: \"b14167e9-c982-44ef-832c-61d06d82f44b\") "
Sep 17 23:25:18.851218 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.851064    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-model-cache" (OuterVolumeSpecName: "model-cache") pod "b14167e9-c982-44ef-832c-61d06d82f44b" (UID: "b14167e9-c982-44ef-832c-61d06d82f44b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:25:18.851218 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.851075    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b14167e9-c982-44ef-832c-61d06d82f44b" (UID: "b14167e9-c982-44ef-832c-61d06d82f44b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:25:18.851320 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.851280    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-home" (OuterVolumeSpecName: "home") pod "b14167e9-c982-44ef-832c-61d06d82f44b" (UID: "b14167e9-c982-44ef-832c-61d06d82f44b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:25:18.851320 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.851292    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-model-cache\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:18.851320 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.851317    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-tmp-dir\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:18.852886 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.852860    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-dshm" (OuterVolumeSpecName: "dshm") pod "b14167e9-c982-44ef-832c-61d06d82f44b" (UID: "b14167e9-c982-44ef-832c-61d06d82f44b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:25:18.853041 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.852930    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b14167e9-c982-44ef-832c-61d06d82f44b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b14167e9-c982-44ef-832c-61d06d82f44b" (UID: "b14167e9-c982-44ef-832c-61d06d82f44b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:25:18.853312 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.853289    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b14167e9-c982-44ef-832c-61d06d82f44b-kube-api-access-hpkvr" (OuterVolumeSpecName: "kube-api-access-hpkvr") pod "b14167e9-c982-44ef-832c-61d06d82f44b" (UID: "b14167e9-c982-44ef-832c-61d06d82f44b"). InnerVolumeSpecName "kube-api-access-hpkvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:25:18.951642 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.951607    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:18.951773 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.951739    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpkvr\" (UniqueName: \"kubernetes.io/projected/b14167e9-c982-44ef-832c-61d06d82f44b-kube-api-access-hpkvr\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:18.951773 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.951756    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b14167e9-c982-44ef-832c-61d06d82f44b-tls-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:18.951773 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.951768    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-home\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:18.951882 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:18.951779    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b14167e9-c982-44ef-832c-61d06d82f44b-dshm\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:19.046444 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.046413    2558 generic.go:356] "Generic (PLEG): container finished" podID="b14167e9-c982-44ef-832c-61d06d82f44b" containerID="ae0438ab7c4cf1fa047b636260b19f177659963e6d84de3f8ef9fd2f6c878745" exitCode=0
Sep 17 23:25:19.046635 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.046472    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp" event={"ID":"b14167e9-c982-44ef-832c-61d06d82f44b","Type":"ContainerDied","Data":"ae0438ab7c4cf1fa047b636260b19f177659963e6d84de3f8ef9fd2f6c878745"}
Sep 17 23:25:19.046635 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.046485    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"
Sep 17 23:25:19.046635 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.046503    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp" event={"ID":"b14167e9-c982-44ef-832c-61d06d82f44b","Type":"ContainerDied","Data":"8b1b159deac3596bfb1d9a68d7d4560b72410bfb0c3e0bca7757118551125d7d"}
Sep 17 23:25:19.046635 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.046545    2558 scope.go:117] "RemoveContainer" containerID="ae0438ab7c4cf1fa047b636260b19f177659963e6d84de3f8ef9fd2f6c878745"
Sep 17 23:25:19.055706 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.055654    2558 scope.go:117] "RemoveContainer" containerID="ae0438ab7c4cf1fa047b636260b19f177659963e6d84de3f8ef9fd2f6c878745"
Sep 17 23:25:19.055918 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:25:19.055900    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae0438ab7c4cf1fa047b636260b19f177659963e6d84de3f8ef9fd2f6c878745\": container with ID starting with ae0438ab7c4cf1fa047b636260b19f177659963e6d84de3f8ef9fd2f6c878745 not found: ID does not exist" containerID="ae0438ab7c4cf1fa047b636260b19f177659963e6d84de3f8ef9fd2f6c878745"
Sep 17 23:25:19.055967 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.055927    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae0438ab7c4cf1fa047b636260b19f177659963e6d84de3f8ef9fd2f6c878745"} err="failed to get container status \"ae0438ab7c4cf1fa047b636260b19f177659963e6d84de3f8ef9fd2f6c878745\": rpc error: code = NotFound desc = could not find container \"ae0438ab7c4cf1fa047b636260b19f177659963e6d84de3f8ef9fd2f6c878745\": container with ID starting with ae0438ab7c4cf1fa047b636260b19f177659963e6d84de3f8ef9fd2f6c878745 not found: ID does not exist"
Sep 17 23:25:19.072266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.072235    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"]
Sep 17 23:25:19.075022 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.075003    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-983490/simulator-w4-983490-kserve-cf97f97b4-2sbzp"]
Sep 17 23:25:19.518772 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.518740    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"]
Sep 17 23:25:19.519062 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.519033    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b" podUID="9a4659d1-bc4f-4fe9-9572-ac673f28a3be" containerName="maas-api" containerID="cri-o://3ec7abcdb58926c6133f14336f8c11fd81e2196219057269b10df3abbccc850a" gracePeriod=30
Sep 17 23:25:19.570232 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.570197    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:19.706198 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.706169    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:19.785738 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.785715    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:25:19.805724 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.805686    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:19.858658 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.858634    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-metrics-certs\") pod \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") "
Sep 17 23:25:19.858786 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.858770    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcpvj\" (UniqueName: \"kubernetes.io/projected/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-kube-api-access-hcpvj\") pod \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") "
Sep 17 23:25:19.858830 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.858802    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-openshift-service-ca\") pod \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") "
Sep 17 23:25:19.858872 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.858833    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-maas-api-tls\") pod \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\" (UID: \"9a4659d1-bc4f-4fe9-9572-ac673f28a3be\") "
Sep 17 23:25:19.859111 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.859085    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9a4659d1-bc4f-4fe9-9572-ac673f28a3be" (UID: "9a4659d1-bc4f-4fe9-9572-ac673f28a3be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:25:19.860763 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.860742    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9a4659d1-bc4f-4fe9-9572-ac673f28a3be" (UID: "9a4659d1-bc4f-4fe9-9572-ac673f28a3be"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:25:19.860850 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.860821    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-kube-api-access-hcpvj" (OuterVolumeSpecName: "kube-api-access-hcpvj") pod "9a4659d1-bc4f-4fe9-9572-ac673f28a3be" (UID: "9a4659d1-bc4f-4fe9-9572-ac673f28a3be"). InnerVolumeSpecName "kube-api-access-hcpvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:25:19.860891 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.860846    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9a4659d1-bc4f-4fe9-9572-ac673f28a3be" (UID: "9a4659d1-bc4f-4fe9-9572-ac673f28a3be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:25:19.876067 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.876039    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:19.932979 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.932949    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b14167e9-c982-44ef-832c-61d06d82f44b" path="/var/lib/kubelet/pods/b14167e9-c982-44ef-832c-61d06d82f44b/volumes"
Sep 17 23:25:19.959637 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.959614    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcpvj\" (UniqueName: \"kubernetes.io/projected/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-kube-api-access-hcpvj\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:19.959637 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.959635    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:19.959775 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.959648    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:19.959775 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:19.959656    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a4659d1-bc4f-4fe9-9572-ac673f28a3be-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:25:20.052436 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:20.052362    2558 generic.go:356] "Generic (PLEG): container finished" podID="9a4659d1-bc4f-4fe9-9572-ac673f28a3be" containerID="3ec7abcdb58926c6133f14336f8c11fd81e2196219057269b10df3abbccc850a" exitCode=0
Sep 17 23:25:20.052436 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:20.052412    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"
Sep 17 23:25:20.052643 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:20.052455    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b" event={"ID":"9a4659d1-bc4f-4fe9-9572-ac673f28a3be","Type":"ContainerDied","Data":"3ec7abcdb58926c6133f14336f8c11fd81e2196219057269b10df3abbccc850a"}
Sep 17 23:25:20.052643 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:20.052532    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b" event={"ID":"9a4659d1-bc4f-4fe9-9572-ac673f28a3be","Type":"ContainerDied","Data":"8402a80d6a68df78140d49fa9674035ad5b2a4b88921a7688eaa084328071fb9"}
Sep 17 23:25:20.052643 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:20.052569    2558 scope.go:117] "RemoveContainer" containerID="3ec7abcdb58926c6133f14336f8c11fd81e2196219057269b10df3abbccc850a"
Sep 17 23:25:20.063273 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:20.063260    2558 scope.go:117] "RemoveContainer" containerID="3ec7abcdb58926c6133f14336f8c11fd81e2196219057269b10df3abbccc850a"
Sep 17 23:25:20.063527 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:25:20.063497    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ec7abcdb58926c6133f14336f8c11fd81e2196219057269b10df3abbccc850a\": container with ID starting with 3ec7abcdb58926c6133f14336f8c11fd81e2196219057269b10df3abbccc850a not found: ID does not exist" containerID="3ec7abcdb58926c6133f14336f8c11fd81e2196219057269b10df3abbccc850a"
Sep 17 23:25:20.063586 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:20.063535    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec7abcdb58926c6133f14336f8c11fd81e2196219057269b10df3abbccc850a"} err="failed to get container status \"3ec7abcdb58926c6133f14336f8c11fd81e2196219057269b10df3abbccc850a\": rpc error: code = NotFound desc = could not find container \"3ec7abcdb58926c6133f14336f8c11fd81e2196219057269b10df3abbccc850a\": container with ID starting with 3ec7abcdb58926c6133f14336f8c11fd81e2196219057269b10df3abbccc850a not found: ID does not exist"
Sep 17 23:25:20.072841 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:20.072816    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"]
Sep 17 23:25:20.076540 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:20.076507    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-861522-867898b758-g586b"]
Sep 17 23:25:20.208747 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:20.208720    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:20.309300 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:20.309230    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:21.028271 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:21.028241    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:21.932037 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:21.931998    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a4659d1-bc4f-4fe9-9572-ac673f28a3be" path="/var/lib/kubelet/pods/9a4659d1-bc4f-4fe9-9572-ac673f28a3be/volumes"
Sep 17 23:25:21.989718 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:21.989687    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:23.804290 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:23.804255    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:25.870833 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:25.870797    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:25.973785 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:25.973752    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:26.154119 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:26.154038    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:27.731764 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:27.731731    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:29.561862 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:29.561833    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:29.665069 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:29.665023    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:29.763274 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:29.763236    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:40.023894 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:40.023826    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:40.266239 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:40.266208    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:41.057565 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:41.057510    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:41.158017 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:41.157990    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:41.257436 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:41.257410    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:41.438457 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:41.438422    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:44.015924 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:44.015888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:44.249882 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:44.249857    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:44.679526 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:44.679493    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:45.004214 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:45.004174    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:45.233013 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:45.232986    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:46.741266 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:46.741232    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:47.537491 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:47.537454    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:47.835748 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:47.835661    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:47.950707 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:47.950679    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:48.036197 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:48.036168    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:48.141074 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:48.141004    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:48.238005 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:48.237975    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:48.338398 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:48.338361    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:48.576995 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:48.576956    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:48.883614 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:48.883537    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:49.006156 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:49.006125    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:49.662369 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:49.662341    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:50.505173 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.505139    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:50.555292 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555259    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"]
Sep 17 23:25:50.555688 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555670    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb987c4b-f861-46c3-a2de-4c74a918f649" containerName="maas-api"
Sep 17 23:25:50.555803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555690    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb987c4b-f861-46c3-a2de-4c74a918f649" containerName="maas-api"
Sep 17 23:25:50.555803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555718    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de0c4bff-b9bd-4b7c-a1cc-ea50e863619d" containerName="maas-api"
Sep 17 23:25:50.555803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555727    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="de0c4bff-b9bd-4b7c-a1cc-ea50e863619d" containerName="maas-api"
Sep 17 23:25:50.555803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555739    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b14167e9-c982-44ef-832c-61d06d82f44b" containerName="main"
Sep 17 23:25:50.555803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555748    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b14167e9-c982-44ef-832c-61d06d82f44b" containerName="main"
Sep 17 23:25:50.555803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555766    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4659d1-bc4f-4fe9-9572-ac673f28a3be" containerName="maas-api"
Sep 17 23:25:50.555803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555774    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4659d1-bc4f-4fe9-9572-ac673f28a3be" containerName="maas-api"
Sep 17 23:25:50.555803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555783    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c765899-1725-4392-a11c-9098a5f4e452" containerName="maas-api"
Sep 17 23:25:50.555803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555790    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c765899-1725-4392-a11c-9098a5f4e452" containerName="maas-api"
Sep 17 23:25:50.556245 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555868    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a4659d1-bc4f-4fe9-9572-ac673f28a3be" containerName="maas-api"
Sep 17 23:25:50.556245 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555878    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb987c4b-f861-46c3-a2de-4c74a918f649" containerName="maas-api"
Sep 17 23:25:50.556245 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555889    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c765899-1725-4392-a11c-9098a5f4e452" containerName="maas-api"
Sep 17 23:25:50.556245 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555901    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b14167e9-c982-44ef-832c-61d06d82f44b" containerName="main"
Sep 17 23:25:50.556245 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.555909    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="de0c4bff-b9bd-4b7c-a1cc-ea50e863619d" containerName="maas-api"
Sep 17 23:25:50.559170 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.559147    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:50.561595 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.561573    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-7f7059-serving-cert\""
Sep 17 23:25:50.568855 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.568833    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"]
Sep 17 23:25:50.589656 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.589635    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:50.660765 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.660742    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:50.704528 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.704489    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/441ffaa7-26ae-440c-b4a9-0b96c525e30e-openshift-service-ca\") pod \"maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:50.704653 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.704534    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-metrics-certs\") pod \"maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:50.704653 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.704614    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-maas-api-tls\") pod \"maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:50.704780 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.704661    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44m67\" (UniqueName: \"kubernetes.io/projected/441ffaa7-26ae-440c-b4a9-0b96c525e30e-kube-api-access-44m67\") pod \"maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:50.805819 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.805757    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/441ffaa7-26ae-440c-b4a9-0b96c525e30e-openshift-service-ca\") pod \"maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:50.805819 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.805788    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-metrics-certs\") pod \"maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:50.805819 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.805816    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-maas-api-tls\") pod \"maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:50.805979 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.805831    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44m67\" (UniqueName: \"kubernetes.io/projected/441ffaa7-26ae-440c-b4a9-0b96c525e30e-kube-api-access-44m67\") pod \"maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:50.805979 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:25:50.805927    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-serving-cert: secret "maas-api-e2e-explicit-7f7059-serving-cert" not found
Sep 17 23:25:50.806050 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:25:50.806008    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-maas-api-tls podName:441ffaa7-26ae-440c-b4a9-0b96c525e30e nodeName:}" failed. No retries permitted until 2026-09-17 23:25:51.305988193 +0000 UTC m=+1175.486626615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-maas-api-tls") pod "maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47" (UID: "441ffaa7-26ae-440c-b4a9-0b96c525e30e") : secret "maas-api-e2e-explicit-7f7059-serving-cert" not found
Sep 17 23:25:50.806481 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.806463    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/441ffaa7-26ae-440c-b4a9-0b96c525e30e-openshift-service-ca\") pod \"maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:50.808255 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.808235    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-metrics-certs\") pod \"maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:50.814845 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:50.814822    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44m67\" (UniqueName: \"kubernetes.io/projected/441ffaa7-26ae-440c-b4a9-0b96c525e30e-kube-api-access-44m67\") pod \"maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:51.226794 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:51.226750    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:51.310145 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:51.310119    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-maas-api-tls\") pod \"maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:51.312381 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:51.312360    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-maas-api-tls\") pod \"maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:51.340456 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:51.340431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:51.456944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:51.456919    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:51.470505 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:51.470483    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:51.742724 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:51.742701    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"]
Sep 17 23:25:51.744439 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:25:51.744413    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod441ffaa7_26ae_440c_b4a9_0b96c525e30e.slice/crio-678c2bf3fd77ed927f8a08a6b95cfb185ed4e7f1cb7dc0142f0dc54f0c7a0d3d WatchSource:0}: Error finding container 678c2bf3fd77ed927f8a08a6b95cfb185ed4e7f1cb7dc0142f0dc54f0c7a0d3d: Status 404 returned error can't find the container with id 678c2bf3fd77ed927f8a08a6b95cfb185ed4e7f1cb7dc0142f0dc54f0c7a0d3d
Sep 17 23:25:52.229191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:52.229158    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47" event={"ID":"441ffaa7-26ae-440c-b4a9-0b96c525e30e","Type":"ContainerStarted","Data":"678c2bf3fd77ed927f8a08a6b95cfb185ed4e7f1cb7dc0142f0dc54f0c7a0d3d"}
Sep 17 23:25:53.236837 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:53.236799    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47" event={"ID":"441ffaa7-26ae-440c-b4a9-0b96c525e30e","Type":"ContainerStarted","Data":"d70893888a57d8e0cc4ef5a5313ab57a7eb658f7e3a0aa59cedd3ffd20b9ef8e"}
Sep 17 23:25:53.237203 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:53.236950    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:25:53.272650 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:53.272607    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47" podStartSLOduration=2.772000157 podStartE2EDuration="3.272588882s" totalImagesPullingTime="500.588725ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:25:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:25:53.269590926 +0000 UTC m=+1177.450229364" watchObservedRunningTime="2026-09-17 23:25:53.272588882 +0000 UTC m=+1177.453227328"
Sep 17 23:25:56.501904 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:56.501870    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:56.573458 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:56.573429    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:56.656600 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:56.656566    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:56.798051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:56.797967    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:57.227729 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:57.227703    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:57.807262 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:57.807233    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:57.869021 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:57.868996    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:25:59.247592 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:25:59.247567    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:26:03.061074 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:03.061032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:03.163178 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:03.163150    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:03.299675 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:03.299650    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:03.609653 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:03.609618    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:03.871791 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:03.871707    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:09.643647 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:09.643612    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:09.792576 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:09.792539    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:10.662710 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:10.662671    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:11.674154 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:11.674119    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:12.397205 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:12.397164    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:13.178168 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.178137    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:13.255212 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.255175    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"]
Sep 17 23:26:13.258721 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.258701    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:13.262189 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.262165    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-df969c31-serving-cert\""
Sep 17 23:26:13.270087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.270069    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"]
Sep 17 23:26:13.356840 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.356808    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:13.404012 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.403983    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-metrics-certs\") pod \"maas-api-e2e-mt-df969c31-7474577996-swbmn\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:13.404137 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.404020    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/443c19a3-c814-4b5b-a48f-da381baf3401-openshift-service-ca\") pod \"maas-api-e2e-mt-df969c31-7474577996-swbmn\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:13.404137 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.404121    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-maas-api-tls\") pod \"maas-api-e2e-mt-df969c31-7474577996-swbmn\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:13.404250 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.404152    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgss6\" (UniqueName: \"kubernetes.io/projected/443c19a3-c814-4b5b-a48f-da381baf3401-kube-api-access-lgss6\") pod \"maas-api-e2e-mt-df969c31-7474577996-swbmn\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:13.457062 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.457038    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:13.504950 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.504919    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-metrics-certs\") pod \"maas-api-e2e-mt-df969c31-7474577996-swbmn\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:13.505056 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.504962    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/443c19a3-c814-4b5b-a48f-da381baf3401-openshift-service-ca\") pod \"maas-api-e2e-mt-df969c31-7474577996-swbmn\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:13.505056 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.505039    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-maas-api-tls\") pod \"maas-api-e2e-mt-df969c31-7474577996-swbmn\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:13.505176 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.505061    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgss6\" (UniqueName: \"kubernetes.io/projected/443c19a3-c814-4b5b-a48f-da381baf3401-kube-api-access-lgss6\") pod \"maas-api-e2e-mt-df969c31-7474577996-swbmn\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:13.505176 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:13.505161    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-serving-cert: secret "maas-api-e2e-mt-df969c31-serving-cert" not found
Sep 17 23:26:13.505279 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:13.505231    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-maas-api-tls podName:443c19a3-c814-4b5b-a48f-da381baf3401 nodeName:}" failed. No retries permitted until 2026-09-17 23:26:14.005211261 +0000 UTC m=+1198.185849691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-maas-api-tls") pod "maas-api-e2e-mt-df969c31-7474577996-swbmn" (UID: "443c19a3-c814-4b5b-a48f-da381baf3401") : secret "maas-api-e2e-mt-df969c31-serving-cert" not found
Sep 17 23:26:13.505704 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.505682    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/443c19a3-c814-4b5b-a48f-da381baf3401-openshift-service-ca\") pod \"maas-api-e2e-mt-df969c31-7474577996-swbmn\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:13.507467 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.507442    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-metrics-certs\") pod \"maas-api-e2e-mt-df969c31-7474577996-swbmn\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:13.515546 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.515509    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgss6\" (UniqueName: \"kubernetes.io/projected/443c19a3-c814-4b5b-a48f-da381baf3401-kube-api-access-lgss6\") pod \"maas-api-e2e-mt-df969c31-7474577996-swbmn\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:13.720850 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.720778    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:13.908428 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:13.908384    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:14.006674 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:14.006587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:14.009811 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:14.009788    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-maas-api-tls\") pod \"maas-api-e2e-mt-df969c31-7474577996-swbmn\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:14.012094 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:14.012077    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-maas-api-tls\") pod \"maas-api-e2e-mt-df969c31-7474577996-swbmn\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:14.169717 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:14.169690    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:14.447447 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:14.447416    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"]
Sep 17 23:26:14.450281 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:26:14.450253    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod443c19a3_c814_4b5b_a48f_da381baf3401.slice/crio-f64bb2e23feb64eb49c479d4c08abc837d34d9a74f4b3e18cf9c128b1725d3ff WatchSource:0}: Error finding container f64bb2e23feb64eb49c479d4c08abc837d34d9a74f4b3e18cf9c128b1725d3ff: Status 404 returned error can't find the container with id f64bb2e23feb64eb49c479d4c08abc837d34d9a74f4b3e18cf9c128b1725d3ff
Sep 17 23:26:14.533342 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:14.533316    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:15.296932 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:15.296892    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:15.352430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:15.352403    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn" event={"ID":"443c19a3-c814-4b5b-a48f-da381baf3401","Type":"ContainerStarted","Data":"e82d88ee619a54074024aa1c8c078a878ff1f4f789c173b58921965344abf652"}
Sep 17 23:26:15.352603 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:15.352449    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn" event={"ID":"443c19a3-c814-4b5b-a48f-da381baf3401","Type":"ContainerStarted","Data":"f64bb2e23feb64eb49c479d4c08abc837d34d9a74f4b3e18cf9c128b1725d3ff"}
Sep 17 23:26:15.352603 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:15.352487    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:15.371602 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:15.371556    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn" podStartSLOduration=2.063410953 podStartE2EDuration="2.371537749s" totalImagesPullingTime="308.126796ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:26:15.370408108 +0000 UTC m=+1199.551046544" watchObservedRunningTime="2026-09-17 23:26:15.371537749 +0000 UTC m=+1199.552176187"
Sep 17 23:26:21.362635 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:21.362604    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:29.845015 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:29.844984    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:32.929290 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.929258    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"]
Sep 17 23:26:32.934205 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.934186    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:32.937685 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.937543    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-7f7059\"/\"explicit-model-7f7059-kserve-self-signed-certs\""
Sep 17 23:26:32.937685 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.937612    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-7f7059\"/\"default-dockercfg-r24k6\""
Sep 17 23:26:32.937859 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.937548    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-7f7059\"/\"openshift-service-ca.crt\""
Sep 17 23:26:32.937923 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.937881    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-7f7059\"/\"kube-root-ca.crt\""
Sep 17 23:26:32.940999 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.940979    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"]
Sep 17 23:26:32.947770 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.947748    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-model-cache\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:32.947874 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.947811    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-home\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:32.947874 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.947868    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:32.947995 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.947975    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2vk9\" (UniqueName: \"kubernetes.io/projected/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-kube-api-access-c2vk9\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:32.948052 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.948033    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-dshm\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:32.948110 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.948060    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tmp-dir\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:32.950941 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:32.950916    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:33.049003 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.048973    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:33.049140 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.049034    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2vk9\" (UniqueName: \"kubernetes.io/projected/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-kube-api-access-c2vk9\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:33.049140 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.049112    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-dshm\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:33.049217 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.049143    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tmp-dir\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:33.049217 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.049164    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-model-cache\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:33.049320 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.049298    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-home\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:33.049579 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.049560    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-model-cache\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:33.049672 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.049612    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tmp-dir\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:33.049672 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.049621    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-home\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:33.051309 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.051288    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-dshm\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:33.051397 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.051372    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:33.057195 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.057178    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2vk9\" (UniqueName: \"kubernetes.io/projected/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-kube-api-access-c2vk9\") pod \"explicit-model-7f7059-kserve-5f7b744557-7vlb6\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") " pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:33.244676 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.244648    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:33.514092 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:33.514020    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"]
Sep 17 23:26:33.516961 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:26:33.516923    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84c48048_7ae5_42f2_aec0_7dd2e24e4f0e.slice/crio-a7ac90ec7d94e2d54e512b4ec660c14fae6f82bdb47c07d5fc381cb4c139bcd1 WatchSource:0}: Error finding container a7ac90ec7d94e2d54e512b4ec660c14fae6f82bdb47c07d5fc381cb4c139bcd1: Status 404 returned error can't find the container with id a7ac90ec7d94e2d54e512b4ec660c14fae6f82bdb47c07d5fc381cb4c139bcd1
Sep 17 23:26:34.448061 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:34.448021    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6" event={"ID":"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e","Type":"ContainerStarted","Data":"9dafa2b5473350140ee6a0f7a3873644bddaa1c7cd5e393b2573923bdae3c6f8"}
Sep 17 23:26:34.448061 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:34.448072    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6" event={"ID":"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e","Type":"ContainerStarted","Data":"a7ac90ec7d94e2d54e512b4ec660c14fae6f82bdb47c07d5fc381cb4c139bcd1"}
Sep 17 23:26:34.464080 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:34.464031    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6" podStartSLOduration=2.464017374 podStartE2EDuration="2.464017374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:26:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:26:34.461841947 +0000 UTC m=+1218.642480386" watchObservedRunningTime="2026-09-17 23:26:34.464017374 +0000 UTC m=+1218.644655812"
Sep 17 23:26:34.783223 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:34.783135    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:35.412392 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:35.412354    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"]
Sep 17 23:26:35.427149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:35.427124    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:35.571231 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:35.571195    2558 secret.go:189] Couldn't get secret ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-self-signed-certs: secret "explicit-model-7f7059-kserve-self-signed-certs" not found
Sep 17 23:26:35.571677 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:35.571274    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs podName:84c48048-7ae5-42f2-aec0-7dd2e24e4f0e nodeName:}" failed. No retries permitted until 2026-09-17 23:26:36.071258924 +0000 UTC m=+1220.251897340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs") pod "explicit-model-7f7059-kserve-5f7b744557-7vlb6" (UID: "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e") : secret "explicit-model-7f7059-kserve-self-signed-certs" not found
Sep 17 23:26:36.063452 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:36.063425    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:36.074662 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:36.074635    2558 secret.go:189] Couldn't get secret ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-self-signed-certs: secret "explicit-model-7f7059-kserve-self-signed-certs" not found
Sep 17 23:26:36.074803 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:36.074709    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs podName:84c48048-7ae5-42f2-aec0-7dd2e24e4f0e nodeName:}" failed. No retries permitted until 2026-09-17 23:26:37.074694207 +0000 UTC m=+1221.255332751 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs") pod "explicit-model-7f7059-kserve-5f7b744557-7vlb6" (UID: "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e") : secret "explicit-model-7f7059-kserve-self-signed-certs" not found
Sep 17 23:26:36.774946 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:36.774909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:37.083921 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:37.083798    2558 secret.go:189] Couldn't get secret ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-self-signed-certs: secret "explicit-model-7f7059-kserve-self-signed-certs" not found
Sep 17 23:26:37.083921 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:37.083879    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs podName:84c48048-7ae5-42f2-aec0-7dd2e24e4f0e nodeName:}" failed. No retries permitted until 2026-09-17 23:26:39.083860425 +0000 UTC m=+1223.264498856 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs") pod "explicit-model-7f7059-kserve-5f7b744557-7vlb6" (UID: "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e") : secret "explicit-model-7f7059-kserve-self-signed-certs" not found
Sep 17 23:26:37.492779 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:37.492744    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:38.387738 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:38.387708    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:39.101319 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:39.101281    2558 secret.go:189] Couldn't get secret ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-self-signed-certs: secret "explicit-model-7f7059-kserve-self-signed-certs" not found
Sep 17 23:26:39.101537 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:39.101353    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs podName:84c48048-7ae5-42f2-aec0-7dd2e24e4f0e nodeName:}" failed. No retries permitted until 2026-09-17 23:26:43.101336544 +0000 UTC m=+1227.281974981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs") pod "explicit-model-7f7059-kserve-5f7b744557-7vlb6" (UID: "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e") : secret "explicit-model-7f7059-kserve-self-signed-certs" not found
Sep 17 23:26:39.165144 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:39.165118    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:39.731319 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:39.731287    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:40.405991 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:40.405957    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:40.505012 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:40.504978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:40.845392 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:40.845364    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:43.134890 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:43.134862    2558 secret.go:189] Couldn't get secret ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-self-signed-certs: secret "explicit-model-7f7059-kserve-self-signed-certs" not found
Sep 17 23:26:43.135234 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:43.134934    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs podName:84c48048-7ae5-42f2-aec0-7dd2e24e4f0e nodeName:}" failed. No retries permitted until 2026-09-17 23:26:51.13491996 +0000 UTC m=+1235.315558376 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs") pod "explicit-model-7f7059-kserve-5f7b744557-7vlb6" (UID: "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e") : secret "explicit-model-7f7059-kserve-self-signed-certs" not found
Sep 17 23:26:43.244982 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:43.244951    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:45.762909 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:45.762878    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:46.332893 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.332853    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"]
Sep 17 23:26:46.333138 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.333112    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47" podUID="441ffaa7-26ae-440c-b4a9-0b96c525e30e" containerName="maas-api" containerID="cri-o://d70893888a57d8e0cc4ef5a5313ab57a7eb658f7e3a0aa59cedd3ffd20b9ef8e" gracePeriod=30
Sep 17 23:26:46.357110 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.357077    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:46.440505 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.440478    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:46.498384 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.498099    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:46.513472 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.513448    2558 generic.go:356] "Generic (PLEG): container finished" podID="441ffaa7-26ae-440c-b4a9-0b96c525e30e" containerID="d70893888a57d8e0cc4ef5a5313ab57a7eb658f7e3a0aa59cedd3ffd20b9ef8e" exitCode=0
Sep 17 23:26:46.513622 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.513497    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47" event={"ID":"441ffaa7-26ae-440c-b4a9-0b96c525e30e","Type":"ContainerDied","Data":"d70893888a57d8e0cc4ef5a5313ab57a7eb658f7e3a0aa59cedd3ffd20b9ef8e"}
Sep 17 23:26:46.557140 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.557111    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:46.583923 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.583864    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:26:46.623910 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.623863    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:46.666926 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.666897    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-metrics-certs\") pod \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") "
Sep 17 23:26:46.666926 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.666931    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-maas-api-tls\") pod \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") "
Sep 17 23:26:46.667133 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.666965    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/441ffaa7-26ae-440c-b4a9-0b96c525e30e-openshift-service-ca\") pod \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") "
Sep 17 23:26:46.667133 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.667008    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44m67\" (UniqueName: \"kubernetes.io/projected/441ffaa7-26ae-440c-b4a9-0b96c525e30e-kube-api-access-44m67\") pod \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\" (UID: \"441ffaa7-26ae-440c-b4a9-0b96c525e30e\") "
Sep 17 23:26:46.667365 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.667337    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/441ffaa7-26ae-440c-b4a9-0b96c525e30e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "441ffaa7-26ae-440c-b4a9-0b96c525e30e" (UID: "441ffaa7-26ae-440c-b4a9-0b96c525e30e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:26:46.669130 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.669105    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/441ffaa7-26ae-440c-b4a9-0b96c525e30e-kube-api-access-44m67" (OuterVolumeSpecName: "kube-api-access-44m67") pod "441ffaa7-26ae-440c-b4a9-0b96c525e30e" (UID: "441ffaa7-26ae-440c-b4a9-0b96c525e30e"). InnerVolumeSpecName "kube-api-access-44m67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:26:46.669130 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.669113    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "441ffaa7-26ae-440c-b4a9-0b96c525e30e" (UID: "441ffaa7-26ae-440c-b4a9-0b96c525e30e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:26:46.669251 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.669170    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "441ffaa7-26ae-440c-b4a9-0b96c525e30e" (UID: "441ffaa7-26ae-440c-b4a9-0b96c525e30e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:26:46.767805 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.767774    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:46.767805 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.767801    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/441ffaa7-26ae-440c-b4a9-0b96c525e30e-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:46.767805 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.767813    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44m67\" (UniqueName: \"kubernetes.io/projected/441ffaa7-26ae-440c-b4a9-0b96c525e30e-kube-api-access-44m67\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:46.768346 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.767822    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ffaa7-26ae-440c-b4a9-0b96c525e30e-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:46.911439 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:46.911357    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:47.008213 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:47.008179    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:47.108176 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:47.108129    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:47.520248 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:47.520222    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"
Sep 17 23:26:47.520418 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:47.520219    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47" event={"ID":"441ffaa7-26ae-440c-b4a9-0b96c525e30e","Type":"ContainerDied","Data":"678c2bf3fd77ed927f8a08a6b95cfb185ed4e7f1cb7dc0142f0dc54f0c7a0d3d"}
Sep 17 23:26:47.520418 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:47.520358    2558 scope.go:117] "RemoveContainer" containerID="d70893888a57d8e0cc4ef5a5313ab57a7eb658f7e3a0aa59cedd3ffd20b9ef8e"
Sep 17 23:26:47.544904 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:47.544877    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"]
Sep 17 23:26:47.549137 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:47.549115    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7f7059-6b5f77889d-qpd47"]
Sep 17 23:26:47.863409 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:47.863326    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:47.933153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:47.933115    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="441ffaa7-26ae-440c-b4a9-0b96c525e30e" path="/var/lib/kubelet/pods/441ffaa7-26ae-440c-b4a9-0b96c525e30e/volumes"
Sep 17 23:26:48.531063 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:48.531023    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:49.349757 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:49.349726    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:50.018230 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:50.018197    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:51.207833 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:51.207804    2558 secret.go:189] Couldn't get secret ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-self-signed-certs: secret "explicit-model-7f7059-kserve-self-signed-certs" not found
Sep 17 23:26:51.208200 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:51.207878    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs podName:84c48048-7ae5-42f2-aec0-7dd2e24e4f0e nodeName:}" failed. No retries permitted until 2026-09-17 23:27:07.207864789 +0000 UTC m=+1251.388503221 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs") pod "explicit-model-7f7059-kserve-5f7b744557-7vlb6" (UID: "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e") : secret "explicit-model-7f7059-kserve-self-signed-certs" not found
Sep 17 23:26:51.477403 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.477322    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6" podUID="84c48048-7ae5-42f2-aec0-7dd2e24e4f0e" containerName="main" containerID="cri-o://9dafa2b5473350140ee6a0f7a3873644bddaa1c7cd5e393b2573923bdae3c6f8" gracePeriod=45
Sep 17 23:26:51.711893 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.711871    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:51.813442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.813382    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-dshm\") pod \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") "
Sep 17 23:26:51.813442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.813410    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-home\") pod \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") "
Sep 17 23:26:51.813442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.813428    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tmp-dir\") pod \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") "
Sep 17 23:26:51.813708 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.813456    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-model-cache\") pod \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") "
Sep 17 23:26:51.813708 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.813482    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2vk9\" (UniqueName: \"kubernetes.io/projected/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-kube-api-access-c2vk9\") pod \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") "
Sep 17 23:26:51.813708 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.813576    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs\") pod \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\" (UID: \"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e\") "
Sep 17 23:26:51.813855 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.813727    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e" (UID: "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:26:51.813855 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.813742    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-home" (OuterVolumeSpecName: "home") pod "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e" (UID: "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:26:51.813855 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.813758    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-model-cache" (OuterVolumeSpecName: "model-cache") pod "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e" (UID: "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:26:51.814018 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.813947    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-home\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:51.814018 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.813962    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tmp-dir\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:51.814018 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.813972    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-model-cache\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:51.815582 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.815549    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e" (UID: "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:26:51.815677 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.815637    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-dshm" (OuterVolumeSpecName: "dshm") pod "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e" (UID: "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:26:51.815677 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.815644    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-kube-api-access-c2vk9" (OuterVolumeSpecName: "kube-api-access-c2vk9") pod "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e" (UID: "84c48048-7ae5-42f2-aec0-7dd2e24e4f0e"). InnerVolumeSpecName "kube-api-access-c2vk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:26:51.915324 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.915297    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-dshm\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:51.915324 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.915321    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2vk9\" (UniqueName: \"kubernetes.io/projected/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-kube-api-access-c2vk9\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:51.915451 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:51.915335    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e-tls-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:52.088955 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:52.088886    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:52.547735 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:52.547704    2558 generic.go:356] "Generic (PLEG): container finished" podID="84c48048-7ae5-42f2-aec0-7dd2e24e4f0e" containerID="9dafa2b5473350140ee6a0f7a3873644bddaa1c7cd5e393b2573923bdae3c6f8" exitCode=0
Sep 17 23:26:52.548200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:52.547755    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"
Sep 17 23:26:52.548200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:52.547794    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6" event={"ID":"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e","Type":"ContainerDied","Data":"9dafa2b5473350140ee6a0f7a3873644bddaa1c7cd5e393b2573923bdae3c6f8"}
Sep 17 23:26:52.548200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:52.547842    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6" event={"ID":"84c48048-7ae5-42f2-aec0-7dd2e24e4f0e","Type":"ContainerDied","Data":"a7ac90ec7d94e2d54e512b4ec660c14fae6f82bdb47c07d5fc381cb4c139bcd1"}
Sep 17 23:26:52.548200 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:52.547867    2558 scope.go:117] "RemoveContainer" containerID="9dafa2b5473350140ee6a0f7a3873644bddaa1c7cd5e393b2573923bdae3c6f8"
Sep 17 23:26:52.556667 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:52.556649    2558 scope.go:117] "RemoveContainer" containerID="9dafa2b5473350140ee6a0f7a3873644bddaa1c7cd5e393b2573923bdae3c6f8"
Sep 17 23:26:52.556945 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:52.556925    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dafa2b5473350140ee6a0f7a3873644bddaa1c7cd5e393b2573923bdae3c6f8\": container with ID starting with 9dafa2b5473350140ee6a0f7a3873644bddaa1c7cd5e393b2573923bdae3c6f8 not found: ID does not exist" containerID="9dafa2b5473350140ee6a0f7a3873644bddaa1c7cd5e393b2573923bdae3c6f8"
Sep 17 23:26:52.557006 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:52.556954    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dafa2b5473350140ee6a0f7a3873644bddaa1c7cd5e393b2573923bdae3c6f8"} err="failed to get container status \"9dafa2b5473350140ee6a0f7a3873644bddaa1c7cd5e393b2573923bdae3c6f8\": rpc error: code = NotFound desc = could not find container \"9dafa2b5473350140ee6a0f7a3873644bddaa1c7cd5e393b2573923bdae3c6f8\": container with ID starting with 9dafa2b5473350140ee6a0f7a3873644bddaa1c7cd5e393b2573923bdae3c6f8 not found: ID does not exist"
Sep 17 23:26:52.578906 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:52.578880    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"]
Sep 17 23:26:52.582545 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:52.582509    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-7f7059/explicit-model-7f7059-kserve-5f7b744557-7vlb6"]
Sep 17 23:26:53.687233 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:53.687203    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:53.749499 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:53.749474    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:53.931924 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:53.931895    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84c48048-7ae5-42f2-aec0-7dd2e24e4f0e" path="/var/lib/kubelet/pods/84c48048-7ae5-42f2-aec0-7dd2e24e4f0e/volumes"
Sep 17 23:26:53.945328 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:53.945271    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:54.393912 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.393836    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:54.700161 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.700129    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"]
Sep 17 23:26:54.700673 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.700652    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="441ffaa7-26ae-440c-b4a9-0b96c525e30e" containerName="maas-api"
Sep 17 23:26:54.700673 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.700675    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="441ffaa7-26ae-440c-b4a9-0b96c525e30e" containerName="maas-api"
Sep 17 23:26:54.700860 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.700703    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84c48048-7ae5-42f2-aec0-7dd2e24e4f0e" containerName="main"
Sep 17 23:26:54.700860 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.700712    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="84c48048-7ae5-42f2-aec0-7dd2e24e4f0e" containerName="main"
Sep 17 23:26:54.700860 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.700820    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="441ffaa7-26ae-440c-b4a9-0b96c525e30e" containerName="maas-api"
Sep 17 23:26:54.700860 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.700831    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="84c48048-7ae5-42f2-aec0-7dd2e24e4f0e" containerName="main"
Sep 17 23:26:54.705485 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.705466    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.708019 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.708002    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-fc8f1e-openshift-default-dockercfg-7kw8n\""
Sep 17 23:26:54.717111 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.717090    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"]
Sep 17 23:26:54.743187 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.743159    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:54.841934 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.841907    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-credential-socket\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.841934 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.841935    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istiod-ca-cert\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.842171 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.841983    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-envoy\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.842171 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.842033    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-podinfo\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.842171 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.842062    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-token\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.842171 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.842098    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-data\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.842171 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.842128    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-workload-socket\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.842171 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.842155    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xlkb\" (UniqueName: \"kubernetes.io/projected/c52d340d-910b-4e3d-a1a9-b0761b328c9c-kube-api-access-9xlkb\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.842480 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.842206    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-workload-certs\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.942803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.942778    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-podinfo\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.942944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.942806    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-token\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.942944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.942827    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-data\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.942944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.942852    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-workload-socket\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.942944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.942876    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xlkb\" (UniqueName: \"kubernetes.io/projected/c52d340d-910b-4e3d-a1a9-b0761b328c9c-kube-api-access-9xlkb\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.942944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.942916    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-workload-certs\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.942944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.942939    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-credential-socket\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.943271 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.942962    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istiod-ca-cert\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.943271 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.943030    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-envoy\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.943271 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.943229    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-data\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.943430 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.943317    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-workload-socket\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.943559 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.943536    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-workload-certs\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.943640 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.943592    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-credential-socket\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.943737 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.943720    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istiod-ca-cert\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.945237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.945215    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-podinfo\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.945348 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.945332    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-envoy\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.952244 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.952186    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xlkb\" (UniqueName: \"kubernetes.io/projected/c52d340d-910b-4e3d-a1a9-b0761b328c9c-kube-api-access-9xlkb\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:54.953082 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:54.953063    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-token\") pod \"e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") " pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:55.016731 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:55.016712    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:55.295293 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:55.295261    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"]
Sep 17 23:26:55.296919 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:26:55.296893    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc52d340d_910b_4e3d_a1a9_b0761b328c9c.slice/crio-7226db9190d8d110120d966e8866af6294e1f9defb541d1a784911e701232076 WatchSource:0}: Error finding container 7226db9190d8d110120d966e8866af6294e1f9defb541d1a784911e701232076: Status 404 returned error can't find the container with id 7226db9190d8d110120d966e8866af6294e1f9defb541d1a784911e701232076
Sep 17 23:26:55.299212 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:55.299185    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:26:55.299271 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:55.299250    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:26:55.299306 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:55.299276    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:26:55.357720 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:55.357693    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:55.566214 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:55.566122    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk" event={"ID":"c52d340d-910b-4e3d-a1a9-b0761b328c9c","Type":"ContainerStarted","Data":"b95bfe19546bedff9d4a11eba37334dc9c7930d76b4767c649f63605f9c1e2bd"}
Sep 17 23:26:55.566214 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:55.566190    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk" event={"ID":"c52d340d-910b-4e3d-a1a9-b0761b328c9c","Type":"ContainerStarted","Data":"7226db9190d8d110120d966e8866af6294e1f9defb541d1a784911e701232076"}
Sep 17 23:26:55.590179 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:55.590127    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk" podStartSLOduration=1.590112511 podStartE2EDuration="1.590112511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:26:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:26:55.587618149 +0000 UTC m=+1239.768256598" watchObservedRunningTime="2026-09-17 23:26:55.590112511 +0000 UTC m=+1239.770750950"
Sep 17 23:26:55.644433 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:55.644406    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:55.872672 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:55.872587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:55.973697 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:55.973671    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:56.017737 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:56.017709    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:56.022690 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:56.022669    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:56.091293 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:56.091262    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:56.571617 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:56.571587    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:56.572539 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:56.572502    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:26:57.026360 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:57.026329    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:57.502128 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:57.502093    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:58.080550 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.080507    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"]
Sep 17 23:26:58.080975 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.080727    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn" podUID="443c19a3-c814-4b5b-a48f-da381baf3401" containerName="maas-api" containerID="cri-o://e82d88ee619a54074024aa1c8c078a878ff1f4f789c173b58921965344abf652" gracePeriod=30
Sep 17 23:26:58.109750 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.109724    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:58.208202 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.208171    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:58.311675 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.311649    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:58.322947 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.322928    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:58.404363 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.404297    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:58.473629 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.472804    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgss6\" (UniqueName: \"kubernetes.io/projected/443c19a3-c814-4b5b-a48f-da381baf3401-kube-api-access-lgss6\") pod \"443c19a3-c814-4b5b-a48f-da381baf3401\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") "
Sep 17 23:26:58.473629 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.472856    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/443c19a3-c814-4b5b-a48f-da381baf3401-openshift-service-ca\") pod \"443c19a3-c814-4b5b-a48f-da381baf3401\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") "
Sep 17 23:26:58.473629 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.472897    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-metrics-certs\") pod \"443c19a3-c814-4b5b-a48f-da381baf3401\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") "
Sep 17 23:26:58.473629 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.473008    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-maas-api-tls\") pod \"443c19a3-c814-4b5b-a48f-da381baf3401\" (UID: \"443c19a3-c814-4b5b-a48f-da381baf3401\") "
Sep 17 23:26:58.474565 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.474283    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/443c19a3-c814-4b5b-a48f-da381baf3401-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "443c19a3-c814-4b5b-a48f-da381baf3401" (UID: "443c19a3-c814-4b5b-a48f-da381baf3401"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:26:58.477793 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.477754    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "443c19a3-c814-4b5b-a48f-da381baf3401" (UID: "443c19a3-c814-4b5b-a48f-da381baf3401"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:26:58.478008 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.477974    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "443c19a3-c814-4b5b-a48f-da381baf3401" (UID: "443c19a3-c814-4b5b-a48f-da381baf3401"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:26:58.478733 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.478707    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/443c19a3-c814-4b5b-a48f-da381baf3401-kube-api-access-lgss6" (OuterVolumeSpecName: "kube-api-access-lgss6") pod "443c19a3-c814-4b5b-a48f-da381baf3401" (UID: "443c19a3-c814-4b5b-a48f-da381baf3401"). InnerVolumeSpecName "kube-api-access-lgss6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:26:58.527672 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.527644    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:58.574026 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.573992    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:58.574026 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.574025    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgss6\" (UniqueName: \"kubernetes.io/projected/443c19a3-c814-4b5b-a48f-da381baf3401-kube-api-access-lgss6\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:58.574026 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.574034    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/443c19a3-c814-4b5b-a48f-da381baf3401-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:58.574199 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.574043    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/443c19a3-c814-4b5b-a48f-da381baf3401-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:26:58.583051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.583022    2558 generic.go:356] "Generic (PLEG): container finished" podID="443c19a3-c814-4b5b-a48f-da381baf3401" containerID="e82d88ee619a54074024aa1c8c078a878ff1f4f789c173b58921965344abf652" exitCode=0
Sep 17 23:26:58.583140 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.583067    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"
Sep 17 23:26:58.583140 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.583112    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn" event={"ID":"443c19a3-c814-4b5b-a48f-da381baf3401","Type":"ContainerDied","Data":"e82d88ee619a54074024aa1c8c078a878ff1f4f789c173b58921965344abf652"}
Sep 17 23:26:58.583253 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.583158    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn" event={"ID":"443c19a3-c814-4b5b-a48f-da381baf3401","Type":"ContainerDied","Data":"f64bb2e23feb64eb49c479d4c08abc837d34d9a74f4b3e18cf9c128b1725d3ff"}
Sep 17 23:26:58.583253 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.583184    2558 scope.go:117] "RemoveContainer" containerID="e82d88ee619a54074024aa1c8c078a878ff1f4f789c173b58921965344abf652"
Sep 17 23:26:58.594228 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.594195    2558 scope.go:117] "RemoveContainer" containerID="e82d88ee619a54074024aa1c8c078a878ff1f4f789c173b58921965344abf652"
Sep 17 23:26:58.594564 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:26:58.594539    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e82d88ee619a54074024aa1c8c078a878ff1f4f789c173b58921965344abf652\": container with ID starting with e82d88ee619a54074024aa1c8c078a878ff1f4f789c173b58921965344abf652 not found: ID does not exist" containerID="e82d88ee619a54074024aa1c8c078a878ff1f4f789c173b58921965344abf652"
Sep 17 23:26:58.594676 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.594573    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e82d88ee619a54074024aa1c8c078a878ff1f4f789c173b58921965344abf652"} err="failed to get container status \"e82d88ee619a54074024aa1c8c078a878ff1f4f789c173b58921965344abf652\": rpc error: code = NotFound desc = could not find container \"e82d88ee619a54074024aa1c8c078a878ff1f4f789c173b58921965344abf652\": container with ID starting with e82d88ee619a54074024aa1c8c078a878ff1f4f789c173b58921965344abf652 not found: ID does not exist"
Sep 17 23:26:58.610264 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.610240    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"]
Sep 17 23:26:58.614017 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.613997    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-df969c31-7474577996-swbmn"]
Sep 17 23:26:58.668129 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:58.668063    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:59.353861 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:59.353829    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:26:59.931975 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:26:59.931943    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="443c19a3-c814-4b5b-a48f-da381baf3401" path="/var/lib/kubelet/pods/443c19a3-c814-4b5b-a48f-da381baf3401/volumes"
Sep 17 23:27:00.032397 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:00.032316    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:00.709207 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:00.709179    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:04.262786 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:04.262756    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:04.347719 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:04.347671    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:04.520594 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:04.520507    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:04.916821 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:04.916746    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:05.031239 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:05.031211    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:05.124506 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:05.124478    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:07.263030 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:07.262992    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:07.831720 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:07.831688    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"]
Sep 17 23:27:07.831924 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:07.831897    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf" podUID="734c919e-fa4b-422d-9c5a-10bc0b305c23" containerName="maas-api" containerID="cri-o://f5442455b6a773c72cc64719e7c6b133cb450d082153ea5769a9a321e93e3518" gracePeriod=30
Sep 17 23:27:07.902622 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:07.902580    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:07.993896 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:07.993863    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:08.064778 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.064755    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:27:08.122817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.122740    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:08.153776 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.153750    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfx7z\" (UniqueName: \"kubernetes.io/projected/734c919e-fa4b-422d-9c5a-10bc0b305c23-kube-api-access-kfx7z\") pod \"734c919e-fa4b-422d-9c5a-10bc0b305c23\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") "
Sep 17 23:27:08.153915 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.153842    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-maas-api-tls\") pod \"734c919e-fa4b-422d-9c5a-10bc0b305c23\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") "
Sep 17 23:27:08.153915 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.153896    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-metrics-certs\") pod \"734c919e-fa4b-422d-9c5a-10bc0b305c23\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") "
Sep 17 23:27:08.154034 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.153973    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/734c919e-fa4b-422d-9c5a-10bc0b305c23-openshift-service-ca\") pod \"734c919e-fa4b-422d-9c5a-10bc0b305c23\" (UID: \"734c919e-fa4b-422d-9c5a-10bc0b305c23\") "
Sep 17 23:27:08.154681 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.154653    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/734c919e-fa4b-422d-9c5a-10bc0b305c23-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "734c919e-fa4b-422d-9c5a-10bc0b305c23" (UID: "734c919e-fa4b-422d-9c5a-10bc0b305c23"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:27:08.156044 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.156010    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "734c919e-fa4b-422d-9c5a-10bc0b305c23" (UID: "734c919e-fa4b-422d-9c5a-10bc0b305c23"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:27:08.156144 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.156061    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "734c919e-fa4b-422d-9c5a-10bc0b305c23" (UID: "734c919e-fa4b-422d-9c5a-10bc0b305c23"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:27:08.156144 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.156085    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/734c919e-fa4b-422d-9c5a-10bc0b305c23-kube-api-access-kfx7z" (OuterVolumeSpecName: "kube-api-access-kfx7z") pod "734c919e-fa4b-422d-9c5a-10bc0b305c23" (UID: "734c919e-fa4b-422d-9c5a-10bc0b305c23"). InnerVolumeSpecName "kube-api-access-kfx7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:27:08.205452 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.205422    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:08.254816 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.254787    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:08.254816 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.254816    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/734c919e-fa4b-422d-9c5a-10bc0b305c23-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:08.255001 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.254826    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfx7z\" (UniqueName: \"kubernetes.io/projected/734c919e-fa4b-422d-9c5a-10bc0b305c23-kube-api-access-kfx7z\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:08.255001 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.254835    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/734c919e-fa4b-422d-9c5a-10bc0b305c23-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:08.455240 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.455210    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:08.584436 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.584409    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:08.638687 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.638664    2558 generic.go:356] "Generic (PLEG): container finished" podID="734c919e-fa4b-422d-9c5a-10bc0b305c23" containerID="f5442455b6a773c72cc64719e7c6b133cb450d082153ea5769a9a321e93e3518" exitCode=0
Sep 17 23:27:08.638811 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.638705    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"
Sep 17 23:27:08.638811 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.638748    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf" event={"ID":"734c919e-fa4b-422d-9c5a-10bc0b305c23","Type":"ContainerDied","Data":"f5442455b6a773c72cc64719e7c6b133cb450d082153ea5769a9a321e93e3518"}
Sep 17 23:27:08.638811 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.638795    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf" event={"ID":"734c919e-fa4b-422d-9c5a-10bc0b305c23","Type":"ContainerDied","Data":"8305e78ff947c9436580982920b9511e96287f3c081182236e59010c8c4175c0"}
Sep 17 23:27:08.638916 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.638818    2558 scope.go:117] "RemoveContainer" containerID="f5442455b6a773c72cc64719e7c6b133cb450d082153ea5769a9a321e93e3518"
Sep 17 23:27:08.650429 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.650355    2558 scope.go:117] "RemoveContainer" containerID="f5442455b6a773c72cc64719e7c6b133cb450d082153ea5769a9a321e93e3518"
Sep 17 23:27:08.650894 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:27:08.650862    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5442455b6a773c72cc64719e7c6b133cb450d082153ea5769a9a321e93e3518\": container with ID starting with f5442455b6a773c72cc64719e7c6b133cb450d082153ea5769a9a321e93e3518 not found: ID does not exist" containerID="f5442455b6a773c72cc64719e7c6b133cb450d082153ea5769a9a321e93e3518"
Sep 17 23:27:08.651002 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.650905    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5442455b6a773c72cc64719e7c6b133cb450d082153ea5769a9a321e93e3518"} err="failed to get container status \"f5442455b6a773c72cc64719e7c6b133cb450d082153ea5769a9a321e93e3518\": rpc error: code = NotFound desc = could not find container \"f5442455b6a773c72cc64719e7c6b133cb450d082153ea5769a9a321e93e3518\": container with ID starting with f5442455b6a773c72cc64719e7c6b133cb450d082153ea5769a9a321e93e3518 not found: ID does not exist"
Sep 17 23:27:08.665129 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.665106    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"]
Sep 17 23:27:08.669037 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:08.669013    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c363f-68cc8498cd-n4nzf"]
Sep 17 23:27:09.356552 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:09.356502    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:09.931757 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:09.931723    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="734c919e-fa4b-422d-9c5a-10bc0b305c23" path="/var/lib/kubelet/pods/734c919e-fa4b-422d-9c5a-10bc0b305c23/volumes"
Sep 17 23:27:10.856564 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:10.856531    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:10.916714 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:10.916689    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:10.987355 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:10.987325    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:11.116244 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:11.116171    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:11.242668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:11.242640    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:14.144433 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:14.144400    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:14.203604 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:14.203569    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:14.403386 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:14.403307    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:15.063075 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:15.063038    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:15.747253 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:15.747213    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:15.878111 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:15.878056    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:16.535894 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:16.535861    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:17.312226 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:17.312193    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:17.431596 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:17.431570    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:18.239108 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:18.239040    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:18.536835 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:18.536763    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:18.762325 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:18.762293    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:21.765384 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:21.765339    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:21.869440 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:21.869405    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:21.965870 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:21.965839    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:22.578489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:22.578411    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:23.540332 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:23.540285    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:24.223100 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.223068    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"]
Sep 17 23:27:24.223581 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.223566    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="443c19a3-c814-4b5b-a48f-da381baf3401" containerName="maas-api"
Sep 17 23:27:24.223660 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.223583    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="443c19a3-c814-4b5b-a48f-da381baf3401" containerName="maas-api"
Sep 17 23:27:24.223660 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.223601    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="734c919e-fa4b-422d-9c5a-10bc0b305c23" containerName="maas-api"
Sep 17 23:27:24.223660 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.223610    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="734c919e-fa4b-422d-9c5a-10bc0b305c23" containerName="maas-api"
Sep 17 23:27:24.223824 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.223705    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="734c919e-fa4b-422d-9c5a-10bc0b305c23" containerName="maas-api"
Sep 17 23:27:24.223824 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.223716    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="443c19a3-c814-4b5b-a48f-da381baf3401" containerName="maas-api"
Sep 17 23:27:24.231694 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.231660    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:24.235290 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.234729    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-d69250-serving-cert\""
Sep 17 23:27:24.238310 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.238282    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"]
Sep 17 23:27:24.257168 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.257145    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:24.279403 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.279377    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6qn4\" (UniqueName: \"kubernetes.io/projected/e8c59032-063b-40d9-8f0c-cf33117ec60a-kube-api-access-z6qn4\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:24.279507 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.279435    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8c59032-063b-40d9-8f0c-cf33117ec60a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:24.279585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.279546    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:24.279654 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.279632    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:24.357669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.357645    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:24.380927 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.380905    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:24.381023 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.380946    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:24.381023 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.380968    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6qn4\" (UniqueName: \"kubernetes.io/projected/e8c59032-063b-40d9-8f0c-cf33117ec60a-kube-api-access-z6qn4\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:24.381140 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:27:24.381054    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-serving-cert: secret "maas-api-e2e-shared-a-w2-d69250-serving-cert" not found
Sep 17 23:27:24.381140 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:27:24.381119    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-maas-api-tls podName:e8c59032-063b-40d9-8f0c-cf33117ec60a nodeName:}" failed. No retries permitted until 2026-09-17 23:27:24.88109957 +0000 UTC m=+1269.061737985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-maas-api-tls") pod "maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw" (UID: "e8c59032-063b-40d9-8f0c-cf33117ec60a") : secret "maas-api-e2e-shared-a-w2-d69250-serving-cert" not found
Sep 17 23:27:24.381260 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.381137    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8c59032-063b-40d9-8f0c-cf33117ec60a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:24.381863 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.381843    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8c59032-063b-40d9-8f0c-cf33117ec60a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:24.383223 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.383203    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:24.396360 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.393058    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6qn4\" (UniqueName: \"kubernetes.io/projected/e8c59032-063b-40d9-8f0c-cf33117ec60a-kube-api-access-z6qn4\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:24.731536 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.731489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:24.828945 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.828914    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:24.886457 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:24.886431    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:24.886635 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:27:24.886595    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-serving-cert: secret "maas-api-e2e-shared-a-w2-d69250-serving-cert" not found
Sep 17 23:27:24.886695 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:27:24.886670    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-maas-api-tls podName:e8c59032-063b-40d9-8f0c-cf33117ec60a nodeName:}" failed. No retries permitted until 2026-09-17 23:27:25.886651452 +0000 UTC m=+1270.067289882 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-maas-api-tls") pod "maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw" (UID: "e8c59032-063b-40d9-8f0c-cf33117ec60a") : secret "maas-api-e2e-shared-a-w2-d69250-serving-cert" not found
Sep 17 23:27:25.009057 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.008973    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"]
Sep 17 23:27:25.009244 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.009215    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct" podUID="35bc8078-6843-4901-8a3c-e5d4672dc763" containerName="maas-api" containerID="cri-o://de0e38351e64bbb27aa43e692a10cee543879075c38bb30d58da71045a7cdf59" gracePeriod=30
Sep 17 23:27:25.033800 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.033774    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:25.137413 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.137387    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:25.234902 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.234875    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:25.248304 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.248284    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:27:25.289802 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.289749    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-maas-api-tls\") pod \"35bc8078-6843-4901-8a3c-e5d4672dc763\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") "
Sep 17 23:27:25.289802 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.289776    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-metrics-certs\") pod \"35bc8078-6843-4901-8a3c-e5d4672dc763\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") "
Sep 17 23:27:25.289921 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.289811    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35bc8078-6843-4901-8a3c-e5d4672dc763-openshift-service-ca\") pod \"35bc8078-6843-4901-8a3c-e5d4672dc763\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") "
Sep 17 23:27:25.289921 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.289839    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqslm\" (UniqueName: \"kubernetes.io/projected/35bc8078-6843-4901-8a3c-e5d4672dc763-kube-api-access-sqslm\") pod \"35bc8078-6843-4901-8a3c-e5d4672dc763\" (UID: \"35bc8078-6843-4901-8a3c-e5d4672dc763\") "
Sep 17 23:27:25.290169 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.290145    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35bc8078-6843-4901-8a3c-e5d4672dc763-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "35bc8078-6843-4901-8a3c-e5d4672dc763" (UID: "35bc8078-6843-4901-8a3c-e5d4672dc763"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:27:25.291825 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.291801    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "35bc8078-6843-4901-8a3c-e5d4672dc763" (UID: "35bc8078-6843-4901-8a3c-e5d4672dc763"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:27:25.291945 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.291925    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35bc8078-6843-4901-8a3c-e5d4672dc763-kube-api-access-sqslm" (OuterVolumeSpecName: "kube-api-access-sqslm") pod "35bc8078-6843-4901-8a3c-e5d4672dc763" (UID: "35bc8078-6843-4901-8a3c-e5d4672dc763"). InnerVolumeSpecName "kube-api-access-sqslm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:27:25.291945 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.291935    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "35bc8078-6843-4901-8a3c-e5d4672dc763" (UID: "35bc8078-6843-4901-8a3c-e5d4672dc763"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:27:25.303677 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.303654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:25.391217 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.391126    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:25.391217 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.391160    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35bc8078-6843-4901-8a3c-e5d4672dc763-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:25.391217 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.391177    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqslm\" (UniqueName: \"kubernetes.io/projected/35bc8078-6843-4901-8a3c-e5d4672dc763-kube-api-access-sqslm\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:25.391217 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.391193    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35bc8078-6843-4901-8a3c-e5d4672dc763-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:25.664659 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.664578    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:25.729815 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.729787    2558 generic.go:356] "Generic (PLEG): container finished" podID="35bc8078-6843-4901-8a3c-e5d4672dc763" containerID="de0e38351e64bbb27aa43e692a10cee543879075c38bb30d58da71045a7cdf59" exitCode=0
Sep 17 23:27:25.729948 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.729835    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"
Sep 17 23:27:25.729948 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.729860    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct" event={"ID":"35bc8078-6843-4901-8a3c-e5d4672dc763","Type":"ContainerDied","Data":"de0e38351e64bbb27aa43e692a10cee543879075c38bb30d58da71045a7cdf59"}
Sep 17 23:27:25.729948 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.729898    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct" event={"ID":"35bc8078-6843-4901-8a3c-e5d4672dc763","Type":"ContainerDied","Data":"46e893f7f5528e81393a546c5a2efe93f81c66a3d2525afeb9221d60cffa9304"}
Sep 17 23:27:25.729948 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.729919    2558 scope.go:117] "RemoveContainer" containerID="de0e38351e64bbb27aa43e692a10cee543879075c38bb30d58da71045a7cdf59"
Sep 17 23:27:25.739106 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.739090    2558 scope.go:117] "RemoveContainer" containerID="de0e38351e64bbb27aa43e692a10cee543879075c38bb30d58da71045a7cdf59"
Sep 17 23:27:25.739373 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:27:25.739325    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de0e38351e64bbb27aa43e692a10cee543879075c38bb30d58da71045a7cdf59\": container with ID starting with de0e38351e64bbb27aa43e692a10cee543879075c38bb30d58da71045a7cdf59 not found: ID does not exist" containerID="de0e38351e64bbb27aa43e692a10cee543879075c38bb30d58da71045a7cdf59"
Sep 17 23:27:25.739373 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.739351    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de0e38351e64bbb27aa43e692a10cee543879075c38bb30d58da71045a7cdf59"} err="failed to get container status \"de0e38351e64bbb27aa43e692a10cee543879075c38bb30d58da71045a7cdf59\": rpc error: code = NotFound desc = could not find container \"de0e38351e64bbb27aa43e692a10cee543879075c38bb30d58da71045a7cdf59\": container with ID starting with de0e38351e64bbb27aa43e692a10cee543879075c38bb30d58da71045a7cdf59 not found: ID does not exist"
Sep 17 23:27:25.751077 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.751047    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"]
Sep 17 23:27:25.756639 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.756618    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-307a7b-77877c85db-qk8ct"]
Sep 17 23:27:25.894340 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.894312    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:25.896620 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.896592    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:25.931980 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:25.931923    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35bc8078-6843-4901-8a3c-e5d4672dc763" path="/var/lib/kubelet/pods/35bc8078-6843-4901-8a3c-e5d4672dc763/volumes"
Sep 17 23:27:26.048428 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:26.048404    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:26.066312 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:26.066285    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:26.323552 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:26.323530    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"]
Sep 17 23:27:26.326032 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:27:26.326003    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c59032_063b_40d9_8f0c_cf33117ec60a.slice/crio-6dbf93912080ea68f85c37a8b720da5499f7aa87952580c5a8b33f8c2d456db4 WatchSource:0}: Error finding container 6dbf93912080ea68f85c37a8b720da5499f7aa87952580c5a8b33f8c2d456db4: Status 404 returned error can't find the container with id 6dbf93912080ea68f85c37a8b720da5499f7aa87952580c5a8b33f8c2d456db4
Sep 17 23:27:26.731568 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:26.731542    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:26.737572 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:26.737542    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw" event={"ID":"e8c59032-063b-40d9-8f0c-cf33117ec60a","Type":"ContainerStarted","Data":"6dbf93912080ea68f85c37a8b720da5499f7aa87952580c5a8b33f8c2d456db4"}
Sep 17 23:27:27.744756 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:27.744719    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw" event={"ID":"e8c59032-063b-40d9-8f0c-cf33117ec60a","Type":"ContainerStarted","Data":"6a63aecbc20ef81a5adebb13e4efdc4af49955d25b06d261c9fb931e34b4a78b"}
Sep 17 23:27:27.745231 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:27.744830    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:27.762661 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:27.762618    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw" podStartSLOduration=3.390743973 podStartE2EDuration="3.762604392s" totalImagesPullingTime="371.860419ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:27:27.761081374 +0000 UTC m=+1271.941719813" watchObservedRunningTime="2026-09-17 23:27:27.762604392 +0000 UTC m=+1271.943242829"
Sep 17 23:27:28.661612 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:28.661579    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:28.787973 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:28.787932    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"]
Sep 17 23:27:28.799494 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:28.798902    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk" podUID="c52d340d-910b-4e3d-a1a9-b0761b328c9c" containerName="istio-proxy" containerID="cri-o://b95bfe19546bedff9d4a11eba37334dc9c7930d76b4767c649f63605f9c1e2bd" gracePeriod=30
Sep 17 23:27:28.804993 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:28.804968    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:28.995881 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:28.995848    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:29.099564 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:29.099505    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:31.398678 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:31.398633    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:31.460994 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:31.460961    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:31.659455 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:31.659379    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:32.211302 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:32.211275    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:32.515394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:32.515291    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:32.847533 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:32.847446    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:33.755611 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:33.755581    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:27:34.036192 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.036168    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:27:34.060509 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.060483    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-token\") pod \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") "
Sep 17 23:27:34.060669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.060536    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xlkb\" (UniqueName: \"kubernetes.io/projected/c52d340d-910b-4e3d-a1a9-b0761b328c9c-kube-api-access-9xlkb\") pod \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") "
Sep 17 23:27:34.060669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.060639    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-data\") pod \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") "
Sep 17 23:27:34.060799 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.060683    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-podinfo\") pod \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") "
Sep 17 23:27:34.060799 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.060717    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-workload-socket\") pod \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") "
Sep 17 23:27:34.060799 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.060747    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-envoy\") pod \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") "
Sep 17 23:27:34.060799 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.060795    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-credential-socket\") pod \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") "
Sep 17 23:27:34.061003 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.060820    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-workload-certs\") pod \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") "
Sep 17 23:27:34.061003 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.060894    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istiod-ca-cert\") pod \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\" (UID: \"c52d340d-910b-4e3d-a1a9-b0761b328c9c\") "
Sep 17 23:27:34.061003 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.060916    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-data" (OuterVolumeSpecName: "istio-data") pod "c52d340d-910b-4e3d-a1a9-b0761b328c9c" (UID: "c52d340d-910b-4e3d-a1a9-b0761b328c9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:27:34.061003 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.060976    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c52d340d-910b-4e3d-a1a9-b0761b328c9c" (UID: "c52d340d-910b-4e3d-a1a9-b0761b328c9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:27:34.061215 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.061150    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c52d340d-910b-4e3d-a1a9-b0761b328c9c" (UID: "c52d340d-910b-4e3d-a1a9-b0761b328c9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:27:34.061276 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.061222    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-data\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:34.061276 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.061243    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-workload-socket\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:34.061276 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.061257    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-workload-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:34.061431 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.061364    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c52d340d-910b-4e3d-a1a9-b0761b328c9c" (UID: "c52d340d-910b-4e3d-a1a9-b0761b328c9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:27:34.061572 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.061548    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c52d340d-910b-4e3d-a1a9-b0761b328c9c" (UID: "c52d340d-910b-4e3d-a1a9-b0761b328c9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:27:34.062821 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.062796    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-token" (OuterVolumeSpecName: "istio-token") pod "c52d340d-910b-4e3d-a1a9-b0761b328c9c" (UID: "c52d340d-910b-4e3d-a1a9-b0761b328c9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:27:34.063098 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.063082    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c52d340d-910b-4e3d-a1a9-b0761b328c9c-kube-api-access-9xlkb" (OuterVolumeSpecName: "kube-api-access-9xlkb") pod "c52d340d-910b-4e3d-a1a9-b0761b328c9c" (UID: "c52d340d-910b-4e3d-a1a9-b0761b328c9c"). InnerVolumeSpecName "kube-api-access-9xlkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:27:34.063223 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.063197    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c52d340d-910b-4e3d-a1a9-b0761b328c9c" (UID: "c52d340d-910b-4e3d-a1a9-b0761b328c9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:27:34.063695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.063661    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c52d340d-910b-4e3d-a1a9-b0761b328c9c" (UID: "c52d340d-910b-4e3d-a1a9-b0761b328c9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:27:34.161815 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.161785    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istiod-ca-cert\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:34.161815 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.161810    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-token\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:34.161815 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.161819    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xlkb\" (UniqueName: \"kubernetes.io/projected/c52d340d-910b-4e3d-a1a9-b0761b328c9c-kube-api-access-9xlkb\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:34.162022 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.161830    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-podinfo\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:34.162022 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.161840    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-istio-envoy\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:34.162022 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.161850    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52d340d-910b-4e3d-a1a9-b0761b328c9c-credential-socket\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:27:34.178492 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.178449    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:34.780460 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.780423    2558 generic.go:356] "Generic (PLEG): container finished" podID="c52d340d-910b-4e3d-a1a9-b0761b328c9c" containerID="b95bfe19546bedff9d4a11eba37334dc9c7930d76b4767c649f63605f9c1e2bd" exitCode=0
Sep 17 23:27:34.780880 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.780470    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"
Sep 17 23:27:34.780880 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.780486    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk" event={"ID":"c52d340d-910b-4e3d-a1a9-b0761b328c9c","Type":"ContainerDied","Data":"b95bfe19546bedff9d4a11eba37334dc9c7930d76b4767c649f63605f9c1e2bd"}
Sep 17 23:27:34.780880 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.780555    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk" event={"ID":"c52d340d-910b-4e3d-a1a9-b0761b328c9c","Type":"ContainerDied","Data":"7226db9190d8d110120d966e8866af6294e1f9defb541d1a784911e701232076"}
Sep 17 23:27:34.780880 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.780589    2558 scope.go:117] "RemoveContainer" containerID="b95bfe19546bedff9d4a11eba37334dc9c7930d76b4767c649f63605f9c1e2bd"
Sep 17 23:27:34.790170 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.790149    2558 scope.go:117] "RemoveContainer" containerID="b95bfe19546bedff9d4a11eba37334dc9c7930d76b4767c649f63605f9c1e2bd"
Sep 17 23:27:34.790427 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:27:34.790409    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b95bfe19546bedff9d4a11eba37334dc9c7930d76b4767c649f63605f9c1e2bd\": container with ID starting with b95bfe19546bedff9d4a11eba37334dc9c7930d76b4767c649f63605f9c1e2bd not found: ID does not exist" containerID="b95bfe19546bedff9d4a11eba37334dc9c7930d76b4767c649f63605f9c1e2bd"
Sep 17 23:27:34.790484 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.790436    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b95bfe19546bedff9d4a11eba37334dc9c7930d76b4767c649f63605f9c1e2bd"} err="failed to get container status \"b95bfe19546bedff9d4a11eba37334dc9c7930d76b4767c649f63605f9c1e2bd\": rpc error: code = NotFound desc = could not find container \"b95bfe19546bedff9d4a11eba37334dc9c7930d76b4767c649f63605f9c1e2bd\": container with ID starting with b95bfe19546bedff9d4a11eba37334dc9c7930d76b4767c649f63605f9c1e2bd not found: ID does not exist"
Sep 17 23:27:34.802259 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.802239    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"]
Sep 17 23:27:34.806074 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:34.806056    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-fc8f1e-openshift-default-5996ffffcd-w8xwk"]
Sep 17 23:27:35.932137 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:35.932062    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c52d340d-910b-4e3d-a1a9-b0761b328c9c" path="/var/lib/kubelet/pods/c52d340d-910b-4e3d-a1a9-b0761b328c9c/volumes"
Sep 17 23:27:35.993326 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:35.993301    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:36.165031 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:36.165000    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:36.612043 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:36.612009    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:36.928978 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:36.928907    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:37.073078 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.073048    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:37.220424 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.220398    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"]
Sep 17 23:27:37.220921 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.220906    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35bc8078-6843-4901-8a3c-e5d4672dc763" containerName="maas-api"
Sep 17 23:27:37.220978 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.220925    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="35bc8078-6843-4901-8a3c-e5d4672dc763" containerName="maas-api"
Sep 17 23:27:37.220978 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.220946    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c52d340d-910b-4e3d-a1a9-b0761b328c9c" containerName="istio-proxy"
Sep 17 23:27:37.220978 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.220955    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c52d340d-910b-4e3d-a1a9-b0761b328c9c" containerName="istio-proxy"
Sep 17 23:27:37.221073 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.221041    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c52d340d-910b-4e3d-a1a9-b0761b328c9c" containerName="istio-proxy"
Sep 17 23:27:37.221073 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.221054    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="35bc8078-6843-4901-8a3c-e5d4672dc763" containerName="maas-api"
Sep 17 23:27:37.224224 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.224205    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.226399 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.226375    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-fd82a9-openshift-default-dockercfg-6ldvs\""
Sep 17 23:27:37.235186 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.235164    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"]
Sep 17 23:27:37.251140 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.251118    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:37.289462 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.289436    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-workload-socket\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.289592 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.289465    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-workload-certs\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.289592 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.289487    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-credential-socket\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.289592 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.289503    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad4c855e-b934-466e-8170-7abb14179078-istio-token\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.289592 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.289588    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad4c855e-b934-466e-8170-7abb14179078-istio-podinfo\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.289748 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.289666    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-istio-data\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.289748 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.289692    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mp45\" (UniqueName: \"kubernetes.io/projected/ad4c855e-b934-466e-8170-7abb14179078-kube-api-access-8mp45\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.289748 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.289731    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad4c855e-b934-466e-8170-7abb14179078-istiod-ca-cert\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.289864 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.289780    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-istio-envoy\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.390566 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.390532    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad4c855e-b934-466e-8170-7abb14179078-istio-podinfo\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.390736 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.390647    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-istio-data\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.390736 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.390674    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mp45\" (UniqueName: \"kubernetes.io/projected/ad4c855e-b934-466e-8170-7abb14179078-kube-api-access-8mp45\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.390736 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.390710    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad4c855e-b934-466e-8170-7abb14179078-istiod-ca-cert\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.390907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.390742    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-istio-envoy\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.390907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.390769    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-workload-socket\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.390907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.390794    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-workload-certs\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.390907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.390853    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-credential-socket\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.390907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.390892    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad4c855e-b934-466e-8170-7abb14179078-istio-token\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.391276 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.391252    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-credential-socket\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.391429 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.391344    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-istio-data\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.391429 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.391402    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-workload-certs\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.391801 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.391775    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad4c855e-b934-466e-8170-7abb14179078-istiod-ca-cert\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.391908 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.391886    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-workload-socket\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.393286 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.393265    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-istio-envoy\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.393411 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.393268    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad4c855e-b934-466e-8170-7abb14179078-istio-podinfo\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.399242 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.399214    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad4c855e-b934-466e-8170-7abb14179078-istio-token\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.399487 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.399467    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mp45\" (UniqueName: \"kubernetes.io/projected/ad4c855e-b934-466e-8170-7abb14179078-kube-api-access-8mp45\") pod \"e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") " pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.430615 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.430590    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:37.535670 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.535611    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:37.808070 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.808046    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"]
Sep 17 23:27:37.809928 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:27:37.809903    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad4c855e_b934_466e_8170_7abb14179078.slice/crio-2aa2444b0ee6e570475b18fbb6f5f137822841c3ddc5109c1b7dd38b90221237 WatchSource:0}: Error finding container 2aa2444b0ee6e570475b18fbb6f5f137822841c3ddc5109c1b7dd38b90221237: Status 404 returned error can't find the container with id 2aa2444b0ee6e570475b18fbb6f5f137822841c3ddc5109c1b7dd38b90221237
Sep 17 23:27:37.812208 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.812174    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:27:37.812281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.812253    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:27:37.812318 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.812296    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:27:37.943495 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.943461    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"]
Sep 17 23:27:37.947289 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.947260    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:37.949392 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.949373    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-d90df4-serving-cert\""
Sep 17 23:27:37.959169 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.959146    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"]
Sep 17 23:27:37.981533 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.981490    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:37.996655 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.996631    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4526w\" (UniqueName: \"kubernetes.io/projected/fea943a5-44f2-4f4c-9270-6b5c0b716731-kube-api-access-4526w\") pod \"maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:37.996775 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.996685    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:37.996775 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.996747    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fea943a5-44f2-4f4c-9270-6b5c0b716731-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:37.996901 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:37.996779    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:38.082282 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.082198    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:38.097459 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.097432    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fea943a5-44f2-4f4c-9270-6b5c0b716731-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:38.097623 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.097472    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:38.097623 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.097538    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4526w\" (UniqueName: \"kubernetes.io/projected/fea943a5-44f2-4f4c-9270-6b5c0b716731-kube-api-access-4526w\") pod \"maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:38.098194 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.097727    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:38.098279 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.098148    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fea943a5-44f2-4f4c-9270-6b5c0b716731-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:38.098339 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:27:38.098303    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-serving-cert: secret "maas-api-e2e-shared-a-w3-d90df4-serving-cert" not found
Sep 17 23:27:38.098396 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:27:38.098386    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-maas-api-tls podName:fea943a5-44f2-4f4c-9270-6b5c0b716731 nodeName:}" failed. No retries permitted until 2026-09-17 23:27:38.598365135 +0000 UTC m=+1282.779003567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-maas-api-tls") pod "maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph" (UID: "fea943a5-44f2-4f4c-9270-6b5c0b716731") : secret "maas-api-e2e-shared-a-w3-d90df4-serving-cert" not found
Sep 17 23:27:38.100871 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.100845    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:38.108566 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.108540    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4526w\" (UniqueName: \"kubernetes.io/projected/fea943a5-44f2-4f4c-9270-6b5c0b716731-kube-api-access-4526w\") pod \"maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:38.451413 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.451383    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:38.551248 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.551213    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:38.603008 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.602982    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:38.605667 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.605634    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:38.624042 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.624011    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:38.803361 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.803285    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh" event={"ID":"ad4c855e-b934-466e-8170-7abb14179078","Type":"ContainerStarted","Data":"ed528b43a3048e1bb7f78efedaf3034009d956cd3a4bd93d43d89ceed04b6553"}
Sep 17 23:27:38.803361 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.803337    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh" event={"ID":"ad4c855e-b934-466e-8170-7abb14179078","Type":"ContainerStarted","Data":"2aa2444b0ee6e570475b18fbb6f5f137822841c3ddc5109c1b7dd38b90221237"}
Sep 17 23:27:38.823368 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.823322    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh" podStartSLOduration=1.823307707 podStartE2EDuration="1.823307707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:27:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:27:38.82125262 +0000 UTC m=+1283.001891058" watchObservedRunningTime="2026-09-17 23:27:38.823307707 +0000 UTC m=+1283.003946144"
Sep 17 23:27:38.859870 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.859850    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:38.893963 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:38.893936    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:39.131010 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:39.130939    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"]
Sep 17 23:27:39.133983 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:27:39.133958    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea943a5_44f2_4f4c_9270_6b5c0b716731.slice/crio-dcee09589943e460b1d6f840c87aab22e34d860d35d488d01b84b6f9ad1201b2 WatchSource:0}: Error finding container dcee09589943e460b1d6f840c87aab22e34d860d35d488d01b84b6f9ad1201b2: Status 404 returned error can't find the container with id dcee09589943e460b1d6f840c87aab22e34d860d35d488d01b84b6f9ad1201b2
Sep 17 23:27:39.536230 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:39.536207    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:39.541086 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:39.541067    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:39.733721 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:39.733690    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:39.809612 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:39.809537    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph" event={"ID":"fea943a5-44f2-4f4c-9270-6b5c0b716731","Type":"ContainerStarted","Data":"db6fc5204d51c3470649dc3ffc437dddf635afc571c5fbca477394aee92c72b0"}
Sep 17 23:27:39.809612 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:39.809594    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph" event={"ID":"fea943a5-44f2-4f4c-9270-6b5c0b716731","Type":"ContainerStarted","Data":"dcee09589943e460b1d6f840c87aab22e34d860d35d488d01b84b6f9ad1201b2"}
Sep 17 23:27:39.809774 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:39.809623    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:39.809774 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:39.809650    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:39.810638 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:39.810618    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:27:39.825560 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:39.825509    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph" podStartSLOduration=2.471248765 podStartE2EDuration="2.825488818s" totalImagesPullingTime="354.240053ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:27:39.824580059 +0000 UTC m=+1284.005218509" watchObservedRunningTime="2026-09-17 23:27:39.825488818 +0000 UTC m=+1284.006127257"
Sep 17 23:27:40.576658 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:40.576627    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:40.881108 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:40.881038    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:40.981751 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:40.981722    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:41.581925 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:41.581895    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:42.486008 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:42.485963    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:43.246714 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.246683    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"]
Sep 17 23:27:43.250240 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.250217    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:43.252465 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.252439    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-fd82a9-serving-cert\""
Sep 17 23:27:43.261117 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.261096    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"]
Sep 17 23:27:43.283826 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.283804    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:43.344701 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.344676    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28461752-d1ab-449e-a630-70cd9eb2d00c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:43.344808 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.344742    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:43.344875 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.344850    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:43.344936 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.344880    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr5wt\" (UniqueName: \"kubernetes.io/projected/28461752-d1ab-449e-a630-70cd9eb2d00c-kube-api-access-lr5wt\") pod \"maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:43.388577 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.388505    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:43.445675 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.445648    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:43.445675 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.445677    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr5wt\" (UniqueName: \"kubernetes.io/projected/28461752-d1ab-449e-a630-70cd9eb2d00c-kube-api-access-lr5wt\") pod \"maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:43.445830 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.445708    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28461752-d1ab-449e-a630-70cd9eb2d00c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:43.445830 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.445757    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:43.445830 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:27:43.445795    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-serving-cert: secret "maas-api-e2e-shared-b-w2-fd82a9-serving-cert" not found
Sep 17 23:27:43.445967 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:27:43.445854    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-maas-api-tls podName:28461752-d1ab-449e-a630-70cd9eb2d00c nodeName:}" failed. No retries permitted until 2026-09-17 23:27:43.945832779 +0000 UTC m=+1288.126471206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-maas-api-tls") pod "maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv" (UID: "28461752-d1ab-449e-a630-70cd9eb2d00c") : secret "maas-api-e2e-shared-b-w2-fd82a9-serving-cert" not found
Sep 17 23:27:43.446579 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.446554    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28461752-d1ab-449e-a630-70cd9eb2d00c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:43.448223 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.448202    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:43.455748 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.455723    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr5wt\" (UniqueName: \"kubernetes.io/projected/28461752-d1ab-449e-a630-70cd9eb2d00c-kube-api-access-lr5wt\") pod \"maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:43.713792 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.713763    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:43.818508 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.818479    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:43.910578 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.910553    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:43.950210 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.950182    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:43.952591 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:43.952557    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:44.164312 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:44.164225    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:44.438490 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:44.438459    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"]
Sep 17 23:27:44.439927 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:27:44.439905    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28461752_d1ab_449e_a630_70cd9eb2d00c.slice/crio-7f9d0b640c10426979b2bb50b4b9ec0a4afae81f58c116e4ab8859c11c3d0a77 WatchSource:0}: Error finding container 7f9d0b640c10426979b2bb50b4b9ec0a4afae81f58c116e4ab8859c11c3d0a77: Status 404 returned error can't find the container with id 7f9d0b640c10426979b2bb50b4b9ec0a4afae81f58c116e4ab8859c11c3d0a77
Sep 17 23:27:44.838343 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:44.838314    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv" event={"ID":"28461752-d1ab-449e-a630-70cd9eb2d00c","Type":"ContainerStarted","Data":"7f9d0b640c10426979b2bb50b4b9ec0a4afae81f58c116e4ab8859c11c3d0a77"}
Sep 17 23:27:45.323833 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:45.323801    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:27:45.821103 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:45.821077    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:27:45.846254 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:45.846226    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv" event={"ID":"28461752-d1ab-449e-a630-70cd9eb2d00c","Type":"ContainerStarted","Data":"65e953e4a4607cc4cfed335b4e270613fcc729821cc79ccae20cb98690969c14"}
Sep 17 23:27:45.846445 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:45.846427    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:45.862967 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:45.862922    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv" podStartSLOduration=2.513496547 podStartE2EDuration="2.862908504s" totalImagesPullingTime="349.411957ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:27:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:27:45.861101594 +0000 UTC m=+1290.041740046" watchObservedRunningTime="2026-09-17 23:27:45.862908504 +0000 UTC m=+1290.043546942"
Sep 17 23:27:51.856459 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:51.856431    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:27:58.965955 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:27:58.965919    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:00.363038 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:00.362923    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:00.936076 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:00.936040    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:01.129567 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:01.129536    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:01.276455 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:01.276388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:01.593339 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:01.593245    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:01.759277 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:01.759245    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:01.859320 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:01.859241    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:01.934887 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:01.934857    2558 generic.go:356] "Generic (PLEG): container finished" podID="ad4c855e-b934-466e-8170-7abb14179078" containerID="ed528b43a3048e1bb7f78efedaf3034009d956cd3a4bd93d43d89ceed04b6553" exitCode=0
Sep 17 23:28:01.935048 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:01.934926    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh" event={"ID":"ad4c855e-b934-466e-8170-7abb14179078","Type":"ContainerDied","Data":"ed528b43a3048e1bb7f78efedaf3034009d956cd3a4bd93d43d89ceed04b6553"}
Sep 17 23:28:01.935364 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:01.935349    2558 scope.go:117] "RemoveContainer" containerID="ed528b43a3048e1bb7f78efedaf3034009d956cd3a4bd93d43d89ceed04b6553"
Sep 17 23:28:01.936401 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:01.936376    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:28:01.936455 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:01.936444    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:28:01.936491 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:01.936478    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:28:02.601149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:02.601114    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:02.756300 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:02.756259    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:02.941672 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:02.941645    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh" event={"ID":"ad4c855e-b934-466e-8170-7abb14179078","Type":"ContainerStarted","Data":"7f7459f40d0e9cb9f7799123d6696d2b992d417e3e23485572e82273132ff7c3"}
Sep 17 23:28:03.536138 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:03.536102    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:28:03.536322 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:03.536278    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 17 23:28:03.536367 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:03.536329    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh" podUID="ad4c855e-b934-466e-8170-7abb14179078" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 17 23:28:03.582906 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:03.582878    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:04.240016 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:04.239962    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:04.337760 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:04.337720    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:04.536392 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:04.536297    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 17 23:28:04.536392 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:04.536356    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh" podUID="ad4c855e-b934-466e-8170-7abb14179078" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 17 23:28:04.823825 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:04.823733    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:05.536850 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:05.536815    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 17 23:28:05.537242 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:05.536871    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh" podUID="ad4c855e-b934-466e-8170-7abb14179078" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 17 23:28:05.562849 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:05.562803    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:05.788735 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:05.788642    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:06.225916 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.225877    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"]
Sep 17 23:28:06.228389 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.228368    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:06.230720 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.230703    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-d92920-serving-cert\""
Sep 17 23:28:06.241160 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.241136    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"]
Sep 17 23:28:06.261919 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.261892    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:06.346351 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.346310    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:06.346573 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.346369    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcd9610-6381-4a07-ad1a-086b475ab60e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:06.346573 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.346423    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66qvl\" (UniqueName: \"kubernetes.io/projected/ffcd9610-6381-4a07-ad1a-086b475ab60e-kube-api-access-66qvl\") pod \"maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:06.346573 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.346501    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:06.358283 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.358252    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:06.447403 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.447366    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:06.447723 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.447684    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcd9610-6381-4a07-ad1a-086b475ab60e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:06.447939 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.447923    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66qvl\" (UniqueName: \"kubernetes.io/projected/ffcd9610-6381-4a07-ad1a-086b475ab60e-kube-api-access-66qvl\") pod \"maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:06.448239 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.448221    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:06.449895 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.449874    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcd9610-6381-4a07-ad1a-086b475ab60e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:06.450117 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:28:06.450092    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-serving-cert: secret "maas-api-e2e-shared-b-w3-d92920-serving-cert" not found
Sep 17 23:28:06.450309 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:28:06.450296    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-maas-api-tls podName:ffcd9610-6381-4a07-ad1a-086b475ab60e nodeName:}" failed. No retries permitted until 2026-09-17 23:28:06.950275546 +0000 UTC m=+1311.130913964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-maas-api-tls") pod "maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5" (UID: "ffcd9610-6381-4a07-ad1a-086b475ab60e") : secret "maas-api-e2e-shared-b-w3-d92920-serving-cert" not found
Sep 17 23:28:06.461238 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.452112    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:06.463159 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.463137    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66qvl\" (UniqueName: \"kubernetes.io/projected/ffcd9610-6381-4a07-ad1a-086b475ab60e-kube-api-access-66qvl\") pod \"maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:06.536838 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.536770    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 17 23:28:06.536967 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.536829    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh" podUID="ad4c855e-b934-466e-8170-7abb14179078" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 17 23:28:06.757193 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.756840    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:06.856578 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.856484    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:06.946727 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.946653    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:06.956897 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.956869    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:06.970368 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:06.970341    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:07.142968 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:07.142883    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:07.167063 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:07.167034    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:07.439603 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:07.439574    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"]
Sep 17 23:28:07.440946 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:28:07.440920    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffcd9610_6381_4a07_ad1a_086b475ab60e.slice/crio-75639c7bfa404e30b9cbe8f229905305a0b19100ccf204d4858511c46938d871 WatchSource:0}: Error finding container 75639c7bfa404e30b9cbe8f229905305a0b19100ccf204d4858511c46938d871: Status 404 returned error can't find the container with id 75639c7bfa404e30b9cbe8f229905305a0b19100ccf204d4858511c46938d871
Sep 17 23:28:07.536912 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:07.536883    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 17 23:28:07.537053 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:07.536933    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh" podUID="ad4c855e-b934-466e-8170-7abb14179078" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 17 23:28:07.977713 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:07.977653    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:07.982828 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:07.982797    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5" event={"ID":"ffcd9610-6381-4a07-ad1a-086b475ab60e","Type":"ContainerStarted","Data":"c0c5b81352a62e427d2ea20c8c78b568684c5d2a1535145a7fc9a7f9b18a4ef9"}
Sep 17 23:28:07.982985 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:07.982853    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5" event={"ID":"ffcd9610-6381-4a07-ad1a-086b475ab60e","Type":"ContainerStarted","Data":"75639c7bfa404e30b9cbe8f229905305a0b19100ccf204d4858511c46938d871"}
Sep 17 23:28:07.982985 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:07.982894    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:07.998049 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:07.997990    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5" podStartSLOduration=1.6610218429999999 podStartE2EDuration="1.997969537s" totalImagesPullingTime="336.947694ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:28:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:28:07.997863205 +0000 UTC m=+1312.178501645" watchObservedRunningTime="2026-09-17 23:28:07.997969537 +0000 UTC m=+1312.178607976"
Sep 17 23:28:08.540373 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:08.540343    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:28:08.540760 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:08.540697    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:28:08.541264 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:08.541246    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:28:08.774866 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:08.774834    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:10.829404 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:10.829368    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:13.992912 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:13.992885    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:28:14.556354 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:14.556308    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:14.863165 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:14.863089    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:15.012212 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:15.012184    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:17.765191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:17.765158    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:18.282253 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:18.282220    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:18.377422 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:18.377392    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:18.848772 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:18.848727    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:19.664777 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:19.664746    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:20.387291 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:20.387260    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:21.098098 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:21.098065    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:21.934483 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:21.934448    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"]
Sep 17 23:28:21.937491 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:21.937474    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:21.939698 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:21.939681    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1b9e74c5-serving-cert\""
Sep 17 23:28:21.954232 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:21.954205    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"]
Sep 17 23:28:21.984936 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:21.984902    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-maas-api-tls\") pod \"maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:21.985104 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:21.984948    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-openshift-service-ca\") pod \"maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:21.985104 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:21.985044    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-metrics-certs\") pod \"maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:21.985223 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:21.985121    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjh99\" (UniqueName: \"kubernetes.io/projected/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-kube-api-access-kjh99\") pod \"maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:22.022718 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.022671    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:22.086415 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.086382    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-maas-api-tls\") pod \"maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:22.086612 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.086420    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-openshift-service-ca\") pod \"maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:22.086612 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.086470    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-metrics-certs\") pod \"maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:22.086612 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:28:22.086558    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-serving-cert: secret "maas-api-e2e-mt-1b9e74c5-serving-cert" not found
Sep 17 23:28:22.086768 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:28:22.086626    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-maas-api-tls podName:e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64 nodeName:}" failed. No retries permitted until 2026-09-17 23:28:22.586605137 +0000 UTC m=+1326.767243570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-maas-api-tls") pod "maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj" (UID: "e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64") : secret "maas-api-e2e-mt-1b9e74c5-serving-cert" not found
Sep 17 23:28:22.086768 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.086563    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjh99\" (UniqueName: \"kubernetes.io/projected/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-kube-api-access-kjh99\") pod \"maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:22.087297 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.087272    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-openshift-service-ca\") pod \"maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:22.088991 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.088970    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-metrics-certs\") pod \"maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:22.095113 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.095088    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjh99\" (UniqueName: \"kubernetes.io/projected/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-kube-api-access-kjh99\") pod \"maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:22.123883 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.123845    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:22.223976 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.223943    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:22.494771 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.494688    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:22.590747 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.590712    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-maas-api-tls\") pod \"maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:22.593246 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.593215    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-maas-api-tls\") pod \"maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:22.593643 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.593603    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:22.854820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.854730    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:22.883410 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:22.883385    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:23.125385 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:23.125346    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"]
Sep 17 23:28:23.126485 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:28:23.126452    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ca681c_b9f0_48c9_ab05_5ba3c4efdf64.slice/crio-c386298dede95fa2b45792449c1855cc809368611cc7c6dfa41cc4188f49c01b WatchSource:0}: Error finding container c386298dede95fa2b45792449c1855cc809368611cc7c6dfa41cc4188f49c01b: Status 404 returned error can't find the container with id c386298dede95fa2b45792449c1855cc809368611cc7c6dfa41cc4188f49c01b
Sep 17 23:28:23.591819 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:23.591788    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:24.063975 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:24.063940    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj" event={"ID":"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64","Type":"ContainerStarted","Data":"4161c627b3a7634d27d7d626c95b32cc221ec08728e0e4bc52a30a7dcef965e4"}
Sep 17 23:28:24.064178 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:24.063993    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj" event={"ID":"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64","Type":"ContainerStarted","Data":"c386298dede95fa2b45792449c1855cc809368611cc7c6dfa41cc4188f49c01b"}
Sep 17 23:28:24.064178 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:24.064020    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:24.081501 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:24.081469    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:24.082241 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:24.082206    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj" podStartSLOduration=2.740087882 podStartE2EDuration="3.082190765s" totalImagesPullingTime="342.102883ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:28:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:28:24.079598482 +0000 UTC m=+1328.260236920" watchObservedRunningTime="2026-09-17 23:28:24.082190765 +0000 UTC m=+1328.262829202"
Sep 17 23:28:24.368546 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:24.368439    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:25.289678 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:25.289641    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:26.066945 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:26.066908    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:27.055201 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:27.055172    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:27.991645 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:27.991600    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:30.074633 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:30.074558    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:28:42.877431 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:42.877393    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:43.766320 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:43.766288    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:44.235840 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:44.235811    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:45.159650 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:45.159616    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:45.875293 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:45.875248    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:46.629609 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:46.629565    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:47.446111 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:47.446067    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:48.214824 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:48.214787    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:28:48.405889 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:28:48.405854    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:04.724406 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:04.724355    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:07.922882 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.922844    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"]
Sep 17 23:29:07.936501 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.935699    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:07.939552 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.939509    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1b9e74c5\"/\"openshift-service-ca.crt\""
Sep 17 23:29:07.939959 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.939937    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1b9e74c5\"/\"e2e-lifecycle-model-1b9e74c5-kserve-self-signed-certs\""
Sep 17 23:29:07.940163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.940142    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1b9e74c5\"/\"default-dockercfg-kfxrk\""
Sep 17 23:29:07.940260 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.940182    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1b9e74c5\"/\"kube-root-ca.crt\""
Sep 17 23:29:07.943474 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.943446    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"]
Sep 17 23:29:07.955462 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.955440    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:07.959753 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.959736    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-home\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:07.959854 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.959769    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-dshm\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:07.959854 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.959808    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcq6h\" (UniqueName: \"kubernetes.io/projected/77c68c9d-305f-44a6-9eab-70bd77e8a508-kube-api-access-mcq6h\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:07.959985 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.959879    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-model-cache\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:07.959985 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.959909    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c68c9d-305f-44a6-9eab-70bd77e8a508-tls-certs\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:07.959985 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:07.959971    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-tmp-dir\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.061070 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.061035    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-tmp-dir\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.061174 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.061092    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-home\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.061174 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.061118    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-dshm\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.061174 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.061154    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcq6h\" (UniqueName: \"kubernetes.io/projected/77c68c9d-305f-44a6-9eab-70bd77e8a508-kube-api-access-mcq6h\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.061174 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.061172    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-model-cache\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.061361 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.061308    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c68c9d-305f-44a6-9eab-70bd77e8a508-tls-certs\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.061491 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.061473    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-tmp-dir\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.061576 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.061539    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-home\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.061636 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.061587    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-model-cache\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.063320 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.063299    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-dshm\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.063684 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.063668    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c68c9d-305f-44a6-9eab-70bd77e8a508-tls-certs\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.068117 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.068098    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcq6h\" (UniqueName: \"kubernetes.io/projected/77c68c9d-305f-44a6-9eab-70bd77e8a508-kube-api-access-mcq6h\") pod \"e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") " pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.247362 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.247333    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:08.524737 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:08.524710    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"]
Sep 17 23:29:08.526305 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:29:08.526281    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77c68c9d_305f_44a6_9eab_70bd77e8a508.slice/crio-48b2305bf5aa5c451b51936d9e2990feaed320fdd100b7a7be078f78e9b9a5b5 WatchSource:0}: Error finding container 48b2305bf5aa5c451b51936d9e2990feaed320fdd100b7a7be078f78e9b9a5b5: Status 404 returned error can't find the container with id 48b2305bf5aa5c451b51936d9e2990feaed320fdd100b7a7be078f78e9b9a5b5
Sep 17 23:29:09.299252 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:09.299217    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8" event={"ID":"77c68c9d-305f-44a6-9eab-70bd77e8a508","Type":"ContainerStarted","Data":"39b45237910bc486d3c2d538aa4f375152c83104d88a496d40df23b7dd724247"}
Sep 17 23:29:09.299252 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:09.299256    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8" event={"ID":"77c68c9d-305f-44a6-9eab-70bd77e8a508","Type":"ContainerStarted","Data":"48b2305bf5aa5c451b51936d9e2990feaed320fdd100b7a7be078f78e9b9a5b5"}
Sep 17 23:29:09.314871 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:09.314819    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8" podStartSLOduration=2.3148019140000002 podStartE2EDuration="2.314801914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:29:09.314029783 +0000 UTC m=+1373.494668220" watchObservedRunningTime="2026-09-17 23:29:09.314801914 +0000 UTC m=+1373.495440353"
Sep 17 23:29:18.247882 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:18.247844    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:18.248435 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:18.248127    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:18.260245 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:18.260216    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:18.354598 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:18.354576    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:29:49.365353 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:49.365326    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:54.866377 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:54.866340    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:55.058351 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:55.058321    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:55.573934 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:55.573888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:55.739940 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:55.739905    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:55.841594 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:55.841503    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:55.940112 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:55.940084    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:56.670170 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:56.670133    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:57.482181 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:57.482146    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:58.135902 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:58.135875    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:29:58.961492 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:29:58.961459    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:09.305920 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.305885    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"]
Sep 17 23:30:09.306391 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.306126    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj" podUID="e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64" containerName="maas-api" containerID="cri-o://4161c627b3a7634d27d7d626c95b32cc221ec08728e0e4bc52a30a7dcef965e4" gracePeriod=30
Sep 17 23:30:09.332814 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.332789    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:09.431893 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.431861    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:09.531656 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.531620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:09.547274 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.547256    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:30:09.590300 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.590232    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:09.606933 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.606910    2558 generic.go:356] "Generic (PLEG): container finished" podID="e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64" containerID="4161c627b3a7634d27d7d626c95b32cc221ec08728e0e4bc52a30a7dcef965e4" exitCode=0
Sep 17 23:30:09.607024 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.606954    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"
Sep 17 23:30:09.607074 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.606953    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj" event={"ID":"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64","Type":"ContainerDied","Data":"4161c627b3a7634d27d7d626c95b32cc221ec08728e0e4bc52a30a7dcef965e4"}
Sep 17 23:30:09.607074 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.607062    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj" event={"ID":"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64","Type":"ContainerDied","Data":"c386298dede95fa2b45792449c1855cc809368611cc7c6dfa41cc4188f49c01b"}
Sep 17 23:30:09.607148 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.607085    2558 scope.go:117] "RemoveContainer" containerID="4161c627b3a7634d27d7d626c95b32cc221ec08728e0e4bc52a30a7dcef965e4"
Sep 17 23:30:09.615797 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.615781    2558 scope.go:117] "RemoveContainer" containerID="4161c627b3a7634d27d7d626c95b32cc221ec08728e0e4bc52a30a7dcef965e4"
Sep 17 23:30:09.616049 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:30:09.616033    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4161c627b3a7634d27d7d626c95b32cc221ec08728e0e4bc52a30a7dcef965e4\": container with ID starting with 4161c627b3a7634d27d7d626c95b32cc221ec08728e0e4bc52a30a7dcef965e4 not found: ID does not exist" containerID="4161c627b3a7634d27d7d626c95b32cc221ec08728e0e4bc52a30a7dcef965e4"
Sep 17 23:30:09.616145 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.616056    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4161c627b3a7634d27d7d626c95b32cc221ec08728e0e4bc52a30a7dcef965e4"} err="failed to get container status \"4161c627b3a7634d27d7d626c95b32cc221ec08728e0e4bc52a30a7dcef965e4\": rpc error: code = NotFound desc = could not find container \"4161c627b3a7634d27d7d626c95b32cc221ec08728e0e4bc52a30a7dcef965e4\": container with ID starting with 4161c627b3a7634d27d7d626c95b32cc221ec08728e0e4bc52a30a7dcef965e4 not found: ID does not exist"
Sep 17 23:30:09.655408 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.655384    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-maas-api-tls\") pod \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") "
Sep 17 23:30:09.655504 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.655410    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-metrics-certs\") pod \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") "
Sep 17 23:30:09.655504 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.655471    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-openshift-service-ca\") pod \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") "
Sep 17 23:30:09.655639 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.655571    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjh99\" (UniqueName: \"kubernetes.io/projected/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-kube-api-access-kjh99\") pod \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\" (UID: \"e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64\") "
Sep 17 23:30:09.656005 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.655981    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64" (UID: "e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:30:09.657684 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.657657    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64" (UID: "e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:30:09.658278 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.658248    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64" (UID: "e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:30:09.658377 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.658313    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-kube-api-access-kjh99" (OuterVolumeSpecName: "kube-api-access-kjh99") pod "e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64" (UID: "e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64"). InnerVolumeSpecName "kube-api-access-kjh99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:30:09.757394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.757365    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:30:09.757394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.757392    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:30:09.757394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.757401    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:30:09.757635 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.757411    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjh99\" (UniqueName: \"kubernetes.io/projected/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64-kube-api-access-kjh99\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:30:09.932259 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.932234    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"]
Sep 17 23:30:09.933687 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.933668    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b9e74c5-7fcd55f877-dx8qj"]
Sep 17 23:30:09.984659 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:09.984620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:10.726331 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:10.726302    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:11.544378 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:11.544345    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:11.932277 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:11.932202    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64" path="/var/lib/kubelet/pods/e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64/volumes"
Sep 17 23:30:12.226095 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:12.226067    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:12.825582 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:12.825551    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:12.964958 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:12.964931    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:17.794394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:17.794357    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"]
Sep 17 23:30:17.837169 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:17.837143    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:26.342986 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:26.342950    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:26.394961 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:26.394928    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:32.814864 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:32.814809    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8" podUID="77c68c9d-305f-44a6-9eab-70bd77e8a508" containerName="main" containerID="cri-o://39b45237910bc486d3c2d538aa4f375152c83104d88a496d40df23b7dd724247" gracePeriod=45
Sep 17 23:30:33.060529 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.060494    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:30:33.135669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.135601    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-home\") pod \"77c68c9d-305f-44a6-9eab-70bd77e8a508\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") "
Sep 17 23:30:33.135669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.135644    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcq6h\" (UniqueName: \"kubernetes.io/projected/77c68c9d-305f-44a6-9eab-70bd77e8a508-kube-api-access-mcq6h\") pod \"77c68c9d-305f-44a6-9eab-70bd77e8a508\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") "
Sep 17 23:30:33.135669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.135666    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-dshm\") pod \"77c68c9d-305f-44a6-9eab-70bd77e8a508\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") "
Sep 17 23:30:33.135878 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.135713    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c68c9d-305f-44a6-9eab-70bd77e8a508-tls-certs\") pod \"77c68c9d-305f-44a6-9eab-70bd77e8a508\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") "
Sep 17 23:30:33.135878 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.135734    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-tmp-dir\") pod \"77c68c9d-305f-44a6-9eab-70bd77e8a508\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") "
Sep 17 23:30:33.135878 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.135765    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-model-cache\") pod \"77c68c9d-305f-44a6-9eab-70bd77e8a508\" (UID: \"77c68c9d-305f-44a6-9eab-70bd77e8a508\") "
Sep 17 23:30:33.136019 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.135892    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-home" (OuterVolumeSpecName: "home") pod "77c68c9d-305f-44a6-9eab-70bd77e8a508" (UID: "77c68c9d-305f-44a6-9eab-70bd77e8a508"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:30:33.136019 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.135983    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "77c68c9d-305f-44a6-9eab-70bd77e8a508" (UID: "77c68c9d-305f-44a6-9eab-70bd77e8a508"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:30:33.136132 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.136074    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-model-cache" (OuterVolumeSpecName: "model-cache") pod "77c68c9d-305f-44a6-9eab-70bd77e8a508" (UID: "77c68c9d-305f-44a6-9eab-70bd77e8a508"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:30:33.136188 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.136146    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-home\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:30:33.136188 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.136160    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-tmp-dir\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:30:33.136188 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.136171    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-model-cache\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:30:33.137765 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.137741    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77c68c9d-305f-44a6-9eab-70bd77e8a508-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "77c68c9d-305f-44a6-9eab-70bd77e8a508" (UID: "77c68c9d-305f-44a6-9eab-70bd77e8a508"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:30:33.137879 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.137848    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-dshm" (OuterVolumeSpecName: "dshm") pod "77c68c9d-305f-44a6-9eab-70bd77e8a508" (UID: "77c68c9d-305f-44a6-9eab-70bd77e8a508"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:30:33.137924 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.137911    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77c68c9d-305f-44a6-9eab-70bd77e8a508-kube-api-access-mcq6h" (OuterVolumeSpecName: "kube-api-access-mcq6h") pod "77c68c9d-305f-44a6-9eab-70bd77e8a508" (UID: "77c68c9d-305f-44a6-9eab-70bd77e8a508"). InnerVolumeSpecName "kube-api-access-mcq6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:30:33.237387 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.237361    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcq6h\" (UniqueName: \"kubernetes.io/projected/77c68c9d-305f-44a6-9eab-70bd77e8a508-kube-api-access-mcq6h\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:30:33.237387 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.237386    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c68c9d-305f-44a6-9eab-70bd77e8a508-dshm\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:30:33.237601 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.237395    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c68c9d-305f-44a6-9eab-70bd77e8a508-tls-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:30:33.734036 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.734003    2558 generic.go:356] "Generic (PLEG): container finished" podID="77c68c9d-305f-44a6-9eab-70bd77e8a508" containerID="39b45237910bc486d3c2d538aa4f375152c83104d88a496d40df23b7dd724247" exitCode=0
Sep 17 23:30:33.734231 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.734046    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"
Sep 17 23:30:33.734231 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.734095    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8" event={"ID":"77c68c9d-305f-44a6-9eab-70bd77e8a508","Type":"ContainerDied","Data":"39b45237910bc486d3c2d538aa4f375152c83104d88a496d40df23b7dd724247"}
Sep 17 23:30:33.734231 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.734146    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8" event={"ID":"77c68c9d-305f-44a6-9eab-70bd77e8a508","Type":"ContainerDied","Data":"48b2305bf5aa5c451b51936d9e2990feaed320fdd100b7a7be078f78e9b9a5b5"}
Sep 17 23:30:33.734231 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.734172    2558 scope.go:117] "RemoveContainer" containerID="39b45237910bc486d3c2d538aa4f375152c83104d88a496d40df23b7dd724247"
Sep 17 23:30:33.743293 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.743277    2558 scope.go:117] "RemoveContainer" containerID="39b45237910bc486d3c2d538aa4f375152c83104d88a496d40df23b7dd724247"
Sep 17 23:30:33.743584 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:30:33.743553    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39b45237910bc486d3c2d538aa4f375152c83104d88a496d40df23b7dd724247\": container with ID starting with 39b45237910bc486d3c2d538aa4f375152c83104d88a496d40df23b7dd724247 not found: ID does not exist" containerID="39b45237910bc486d3c2d538aa4f375152c83104d88a496d40df23b7dd724247"
Sep 17 23:30:33.743664 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.743591    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39b45237910bc486d3c2d538aa4f375152c83104d88a496d40df23b7dd724247"} err="failed to get container status \"39b45237910bc486d3c2d538aa4f375152c83104d88a496d40df23b7dd724247\": rpc error: code = NotFound desc = could not find container \"39b45237910bc486d3c2d538aa4f375152c83104d88a496d40df23b7dd724247\": container with ID starting with 39b45237910bc486d3c2d538aa4f375152c83104d88a496d40df23b7dd724247 not found: ID does not exist"
Sep 17 23:30:33.755019 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.755000    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"]
Sep 17 23:30:33.758061 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.758040    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-1b9e74c5/e2e-lifecycle-model-1b9e74c5-kserve-67c8f484cf-qzbw8"]
Sep 17 23:30:33.932330 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:33.932307    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77c68c9d-305f-44a6-9eab-70bd77e8a508" path="/var/lib/kubelet/pods/77c68c9d-305f-44a6-9eab-70bd77e8a508/volumes"
Sep 17 23:30:42.923657 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:42.923623    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:43.696299 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:43.696264    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:49.360411 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:49.360373    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:49.418675 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:49.418649    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:49.615120 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:49.615044    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:50.881886 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:50.881850    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:50.981684 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:50.981652    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:51.083344 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:51.083310    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:51.700739 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:51.700699    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:51.800788 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:51.800758    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:51.901840 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:51.901814    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:53.197385 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:53.197354    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:59.021224 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:59.021188    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:59.088197 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:59.088170    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:30:59.696249 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:30:59.696211    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:05.943750 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:05.943714    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:06.047484 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:06.047056    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:06.143237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:06.143204    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:16.651460 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:16.651427    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:16.881668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:16.881638    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:34.028655 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:34.028579    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:37.151881 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:37.151845    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:37.201053 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:37.201022    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:37.804635 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:37.804605    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:40.724918 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:40.724883    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:41.626795 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:41.626764    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:43.367051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:43.367023    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:43.427030 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:43.426991    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:43.739079 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:43.739052    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:44.055786 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:44.055669    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:44.564639 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:44.564597    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:46.778391 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:46.778350    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:47.656216 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:47.656186    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:47.748399 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:47.748365    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:47.985246 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:47.985216    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:48.581700 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:48.581667    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:49.572257 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:49.572225    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:50.295535 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:50.295465    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:51.129121 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.129088    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"]
Sep 17 23:31:51.129614 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.129493    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77c68c9d-305f-44a6-9eab-70bd77e8a508" containerName="main"
Sep 17 23:31:51.129614 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.129507    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="77c68c9d-305f-44a6-9eab-70bd77e8a508" containerName="main"
Sep 17 23:31:51.129614 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.129541    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64" containerName="maas-api"
Sep 17 23:31:51.129614 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.129547    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64" containerName="maas-api"
Sep 17 23:31:51.129853 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.129620    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2ca681c-b9f0-48c9-ab05-5ba3c4efdf64" containerName="maas-api"
Sep 17 23:31:51.129853 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.129628    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="77c68c9d-305f-44a6-9eab-70bd77e8a508" containerName="main"
Sep 17 23:31:51.132787 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.132767    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.134994 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.134971    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-78432f-serving-cert\""
Sep 17 23:31:51.144709 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.144682    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"]
Sep 17 23:31:51.163481 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.163443    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:51.242173 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.241233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlhpq\" (UniqueName: \"kubernetes.io/projected/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-kube-api-access-rlhpq\") pod \"maas-api-e2e-api-a-78432f-fbbb5d54d-vx962\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.242173 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.241301    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-metrics-certs\") pod \"maas-api-e2e-api-a-78432f-fbbb5d54d-vx962\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.242173 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.241375    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-openshift-service-ca\") pod \"maas-api-e2e-api-a-78432f-fbbb5d54d-vx962\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.242173 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.241473    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-maas-api-tls\") pod \"maas-api-e2e-api-a-78432f-fbbb5d54d-vx962\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.264474 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.264436    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:51.342212 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.342183    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlhpq\" (UniqueName: \"kubernetes.io/projected/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-kube-api-access-rlhpq\") pod \"maas-api-e2e-api-a-78432f-fbbb5d54d-vx962\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.342347 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.342228    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-metrics-certs\") pod \"maas-api-e2e-api-a-78432f-fbbb5d54d-vx962\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.342347 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.342256    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-openshift-service-ca\") pod \"maas-api-e2e-api-a-78432f-fbbb5d54d-vx962\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.342347 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.342314    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-maas-api-tls\") pod \"maas-api-e2e-api-a-78432f-fbbb5d54d-vx962\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.342575 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:31:51.342427    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-serving-cert: secret "maas-api-e2e-api-a-78432f-serving-cert" not found
Sep 17 23:31:51.342575 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:31:51.342506    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-maas-api-tls podName:5c8c7af9-50bb-4b9c-9421-96e656edcfa1 nodeName:}" failed. No retries permitted until 2026-09-17 23:31:51.842478803 +0000 UTC m=+1536.023117221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-maas-api-tls") pod "maas-api-e2e-api-a-78432f-fbbb5d54d-vx962" (UID: "5c8c7af9-50bb-4b9c-9421-96e656edcfa1") : secret "maas-api-e2e-api-a-78432f-serving-cert" not found
Sep 17 23:31:51.342927 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.342908    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-openshift-service-ca\") pod \"maas-api-e2e-api-a-78432f-fbbb5d54d-vx962\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.344661 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.344645    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-metrics-certs\") pod \"maas-api-e2e-api-a-78432f-fbbb5d54d-vx962\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.353132 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.353104    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlhpq\" (UniqueName: \"kubernetes.io/projected/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-kube-api-access-rlhpq\") pod \"maas-api-e2e-api-a-78432f-fbbb5d54d-vx962\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.658664 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.658629    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:51.760979 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.760948    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:51.846425 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.846393    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-maas-api-tls\") pod \"maas-api-e2e-api-a-78432f-fbbb5d54d-vx962\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.848660 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.848641    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-maas-api-tls\") pod \"maas-api-e2e-api-a-78432f-fbbb5d54d-vx962\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:51.859148 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:51.859125    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:31:52.044436 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:52.044404    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:52.313669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:52.313643    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"]
Sep 17 23:31:52.315122 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:31:52.315096    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c8c7af9_50bb_4b9c_9421_96e656edcfa1.slice/crio-06b81146e282e96ddb126483b845691226c5139ca9579edf7ee9b134b16977a8 WatchSource:0}: Error finding container 06b81146e282e96ddb126483b845691226c5139ca9579edf7ee9b134b16977a8: Status 404 returned error can't find the container with id 06b81146e282e96ddb126483b845691226c5139ca9579edf7ee9b134b16977a8
Sep 17 23:31:52.316693 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:52.316674    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:31:53.140764 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:53.140731    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962" event={"ID":"5c8c7af9-50bb-4b9c-9421-96e656edcfa1","Type":"ContainerStarted","Data":"8a50032e4dccca8ed6429fc4425da6ac5d0e9d80602058edc3428e70dc2c2b8c"}
Sep 17 23:31:53.140917 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:53.140786    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962" event={"ID":"5c8c7af9-50bb-4b9c-9421-96e656edcfa1","Type":"ContainerStarted","Data":"06b81146e282e96ddb126483b845691226c5139ca9579edf7ee9b134b16977a8"}
Sep 17 23:31:53.140917 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:53.140844    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:53.161246 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:53.161194    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962" podStartSLOduration=1.456982913 podStartE2EDuration="2.161182025s" totalImagesPullingTime="704.199112ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:31:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:31:53.158231588 +0000 UTC m=+1537.338870039" watchObservedRunningTime="2026-09-17 23:31:53.161182025 +0000 UTC m=+1537.341820463"
Sep 17 23:31:59.151716 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:59.151679    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:31:59.568825 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:31:59.568790    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:08.081907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:08.081856    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:13.388383 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:13.388346    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:14.047180 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:14.047146    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:14.815755 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:14.815720    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:15.476561 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:15.476506    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:16.478300 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:16.478261    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:17.147109 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:17.147078    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:17.708814 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:17.708784    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:18.169311 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:18.169237    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:18.484657 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:18.484607    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:18.584749 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:18.584711    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:21.154747 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:21.154712    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:21.793574 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:21.793536    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:21.892184 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:21.892144    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:22.080100 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:22.080022    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:22.781680 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:22.781644    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:23.639438 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:23.639395    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:24.301913 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:24.301881    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:24.991424 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:24.991391    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:25.741108 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.741076    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"]
Sep 17 23:32:25.744458 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.744430    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:25.746900 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.746876    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-cf8993-serving-cert\""
Sep 17 23:32:25.756258 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.756234    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"]
Sep 17 23:32:25.777209 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.777181    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:25.831905 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.831877    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-metrics-certs\") pod \"maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:25.832006 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.831915    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd35ee7-644c-4a76-be41-78dd6189da8d-openshift-service-ca\") pod \"maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:25.832006 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.831973    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcjdm\" (UniqueName: \"kubernetes.io/projected/bdd35ee7-644c-4a76-be41-78dd6189da8d-kube-api-access-fcjdm\") pod \"maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:25.832094 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.832026    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-maas-api-tls\") pod \"maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:25.834535 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.834494    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:25.932531 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.932491    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-metrics-certs\") pod \"maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:25.932712 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.932542    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd35ee7-644c-4a76-be41-78dd6189da8d-openshift-service-ca\") pod \"maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:25.932788 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.932720    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjdm\" (UniqueName: \"kubernetes.io/projected/bdd35ee7-644c-4a76-be41-78dd6189da8d-kube-api-access-fcjdm\") pod \"maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:25.932856 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.932819    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-maas-api-tls\") pod \"maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:25.932966 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:32:25.932948    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-serving-cert: secret "maas-api-e2e-api-b-cf8993-serving-cert" not found
Sep 17 23:32:25.933046 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:32:25.933021    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-maas-api-tls podName:bdd35ee7-644c-4a76-be41-78dd6189da8d nodeName:}" failed. No retries permitted until 2026-09-17 23:32:26.433001356 +0000 UTC m=+1570.613639775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-maas-api-tls") pod "maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99" (UID: "bdd35ee7-644c-4a76-be41-78dd6189da8d") : secret "maas-api-e2e-api-b-cf8993-serving-cert" not found
Sep 17 23:32:25.933281 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.933258    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd35ee7-644c-4a76-be41-78dd6189da8d-openshift-service-ca\") pod \"maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:25.935119 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.935091    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-metrics-certs\") pod \"maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:25.940725 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:25.940704    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcjdm\" (UniqueName: \"kubernetes.io/projected/bdd35ee7-644c-4a76-be41-78dd6189da8d-kube-api-access-fcjdm\") pod \"maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:26.142090 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:26.142009    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:26.239885 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:26.239859    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:26.341163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:26.341136    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:26.437165 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:26.437125    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-maas-api-tls\") pod \"maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:26.441702 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:26.440870    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-maas-api-tls\") pod \"maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:26.656173 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:26.656139    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:26.927785 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:26.927760    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"]
Sep 17 23:32:26.929414 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:32:26.929392    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdd35ee7_644c_4a76_be41_78dd6189da8d.slice/crio-fc2bb25082afb8791eb2ba37f17b8a9d1e1a23ebf0f31bd331e9f8e9484d48b8 WatchSource:0}: Error finding container fc2bb25082afb8791eb2ba37f17b8a9d1e1a23ebf0f31bd331e9f8e9484d48b8: Status 404 returned error can't find the container with id fc2bb25082afb8791eb2ba37f17b8a9d1e1a23ebf0f31bd331e9f8e9484d48b8
Sep 17 23:32:27.314001 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:27.313969    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99" event={"ID":"bdd35ee7-644c-4a76-be41-78dd6189da8d","Type":"ContainerStarted","Data":"fc2bb25082afb8791eb2ba37f17b8a9d1e1a23ebf0f31bd331e9f8e9484d48b8"}
Sep 17 23:32:28.320244 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:28.320210    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99" event={"ID":"bdd35ee7-644c-4a76-be41-78dd6189da8d","Type":"ContainerStarted","Data":"031ae722e6b57457cd162d0fa810e0127bcf518d71681bdca12fae861f81e05a"}
Sep 17 23:32:28.320657 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:28.320352    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:28.336146 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:28.336102    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99" podStartSLOduration=2.868029521 podStartE2EDuration="3.336086815s" totalImagesPullingTime="468.057294ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:32:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:32:28.334823557 +0000 UTC m=+1572.515461996" watchObservedRunningTime="2026-09-17 23:32:28.336086815 +0000 UTC m=+1572.516725253"
Sep 17 23:32:32.003649 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:32.003611    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:32.056494 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:32.056472    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:34.331005 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:34.330977    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:32:37.860768 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:37.860736    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:46.168528 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:46.168484    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:47.568148 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:47.568114    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:49.100810 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:49.100775    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:50.010233 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:50.010206    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:50.791740 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:50.791697    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:51.668672 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:51.668640    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:51.929976 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:51.929898    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:52.407163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:52.407132    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:53.246134 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:53.246102    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:54.065595 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:54.065563    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:55.051119 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:55.051087    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:55.759075 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:55.759044    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:56.413432 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:56.413401    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:57.175104 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:57.175075    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:57.843450 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:57.843417    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:58.516862 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:58.516829    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:59.279629 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:59.279600    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:32:59.929038 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:32:59.929012    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:00.763077 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:00.763044    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:04.683725 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:04.683681    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:09.013549 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:09.013497    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:09.066753 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:09.066723    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:10.249692 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.249656    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"]
Sep 17 23:33:10.250139 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.249922    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962" podUID="5c8c7af9-50bb-4b9c-9421-96e656edcfa1" containerName="maas-api" containerID="cri-o://8a50032e4dccca8ed6429fc4425da6ac5d0e9d80602058edc3428e70dc2c2b8c" gracePeriod=30
Sep 17 23:33:10.269866 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.269845    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:10.369986 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.369958    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:10.470481 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.470451    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:10.495436 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.495415    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:33:10.535239 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.535177    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:10.540143 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.540112    2558 generic.go:356] "Generic (PLEG): container finished" podID="5c8c7af9-50bb-4b9c-9421-96e656edcfa1" containerID="8a50032e4dccca8ed6429fc4425da6ac5d0e9d80602058edc3428e70dc2c2b8c" exitCode=0
Sep 17 23:33:10.540262 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.540149    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"
Sep 17 23:33:10.540262 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.540173    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962" event={"ID":"5c8c7af9-50bb-4b9c-9421-96e656edcfa1","Type":"ContainerDied","Data":"8a50032e4dccca8ed6429fc4425da6ac5d0e9d80602058edc3428e70dc2c2b8c"}
Sep 17 23:33:10.540262 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.540225    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962" event={"ID":"5c8c7af9-50bb-4b9c-9421-96e656edcfa1","Type":"ContainerDied","Data":"06b81146e282e96ddb126483b845691226c5139ca9579edf7ee9b134b16977a8"}
Sep 17 23:33:10.540262 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.540257    2558 scope.go:117] "RemoveContainer" containerID="8a50032e4dccca8ed6429fc4425da6ac5d0e9d80602058edc3428e70dc2c2b8c"
Sep 17 23:33:10.549131 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.549115    2558 scope.go:117] "RemoveContainer" containerID="8a50032e4dccca8ed6429fc4425da6ac5d0e9d80602058edc3428e70dc2c2b8c"
Sep 17 23:33:10.549389 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:33:10.549371    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a50032e4dccca8ed6429fc4425da6ac5d0e9d80602058edc3428e70dc2c2b8c\": container with ID starting with 8a50032e4dccca8ed6429fc4425da6ac5d0e9d80602058edc3428e70dc2c2b8c not found: ID does not exist" containerID="8a50032e4dccca8ed6429fc4425da6ac5d0e9d80602058edc3428e70dc2c2b8c"
Sep 17 23:33:10.549432 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.549397    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a50032e4dccca8ed6429fc4425da6ac5d0e9d80602058edc3428e70dc2c2b8c"} err="failed to get container status \"8a50032e4dccca8ed6429fc4425da6ac5d0e9d80602058edc3428e70dc2c2b8c\": rpc error: code = NotFound desc = could not find container \"8a50032e4dccca8ed6429fc4425da6ac5d0e9d80602058edc3428e70dc2c2b8c\": container with ID starting with 8a50032e4dccca8ed6429fc4425da6ac5d0e9d80602058edc3428e70dc2c2b8c not found: ID does not exist"
Sep 17 23:33:10.600343 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.600318    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-maas-api-tls\") pod \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") "
Sep 17 23:33:10.600444 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.600366    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-openshift-service-ca\") pod \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") "
Sep 17 23:33:10.600444 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.600399    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-metrics-certs\") pod \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") "
Sep 17 23:33:10.600444 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.600427    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlhpq\" (UniqueName: \"kubernetes.io/projected/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-kube-api-access-rlhpq\") pod \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\" (UID: \"5c8c7af9-50bb-4b9c-9421-96e656edcfa1\") "
Sep 17 23:33:10.600819 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.600792    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5c8c7af9-50bb-4b9c-9421-96e656edcfa1" (UID: "5c8c7af9-50bb-4b9c-9421-96e656edcfa1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:33:10.602319 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.602297    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5c8c7af9-50bb-4b9c-9421-96e656edcfa1" (UID: "5c8c7af9-50bb-4b9c-9421-96e656edcfa1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:33:10.602508 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.602488    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-kube-api-access-rlhpq" (OuterVolumeSpecName: "kube-api-access-rlhpq") pod "5c8c7af9-50bb-4b9c-9421-96e656edcfa1" (UID: "5c8c7af9-50bb-4b9c-9421-96e656edcfa1"). InnerVolumeSpecName "kube-api-access-rlhpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:33:10.602508 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.602504    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5c8c7af9-50bb-4b9c-9421-96e656edcfa1" (UID: "5c8c7af9-50bb-4b9c-9421-96e656edcfa1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:33:10.702022 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.701986    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:33:10.702191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.702029    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlhpq\" (UniqueName: \"kubernetes.io/projected/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-kube-api-access-rlhpq\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:33:10.702191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.702050    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:33:10.702191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.702061    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8c7af9-50bb-4b9c-9421-96e656edcfa1-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:33:10.883873 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.883842    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"]
Sep 17 23:33:10.891826 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.891803    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:10.896870 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:10.896847    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-78432f-fbbb5d54d-vx962"]
Sep 17 23:33:11.004957 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:11.004921    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:11.854297 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:11.854267    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:11.934824 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:11.934793    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c8c7af9-50bb-4b9c-9421-96e656edcfa1" path="/var/lib/kubelet/pods/5c8c7af9-50bb-4b9c-9421-96e656edcfa1/volumes"
Sep 17 23:33:12.550677 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:12.550647    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:13.252676 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:13.252646    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:13.969970 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:13.969938    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:14.613763 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:14.613726    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:16.616677 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:16.616642    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:16.676699 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:16.676665    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:16.888392 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:16.888318    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:18.089256 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:18.089223    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:18.950951 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:18.950919    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:19.616900 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:19.616870    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:20.353247 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:20.353216    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:21.169457 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:21.169426    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:21.823698 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:21.823653    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:28.830434 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:28.830394    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:30.939434 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:30.939382    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:32.036944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:32.036914    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:32.802318 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:32.802288    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"]
Sep 17 23:33:32.802493 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:32.802471    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99" podUID="bdd35ee7-644c-4a76-be41-78dd6189da8d" containerName="maas-api" containerID="cri-o://031ae722e6b57457cd162d0fa810e0127bcf518d71681bdca12fae861f81e05a" gracePeriod=30
Sep 17 23:33:32.829790 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:32.829755    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:32.894827 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:32.894801    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:32.954719 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:32.954684    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:33.029163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.029134    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:33.038591 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.038568    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:33:33.090364 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.090303    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcjdm\" (UniqueName: \"kubernetes.io/projected/bdd35ee7-644c-4a76-be41-78dd6189da8d-kube-api-access-fcjdm\") pod \"bdd35ee7-644c-4a76-be41-78dd6189da8d\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") "
Sep 17 23:33:33.090456 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.090364    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd35ee7-644c-4a76-be41-78dd6189da8d-openshift-service-ca\") pod \"bdd35ee7-644c-4a76-be41-78dd6189da8d\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") "
Sep 17 23:33:33.090456 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.090384    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-maas-api-tls\") pod \"bdd35ee7-644c-4a76-be41-78dd6189da8d\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") "
Sep 17 23:33:33.090570 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.090494    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-metrics-certs\") pod \"bdd35ee7-644c-4a76-be41-78dd6189da8d\" (UID: \"bdd35ee7-644c-4a76-be41-78dd6189da8d\") "
Sep 17 23:33:33.090783 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.090757    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdd35ee7-644c-4a76-be41-78dd6189da8d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bdd35ee7-644c-4a76-be41-78dd6189da8d" (UID: "bdd35ee7-644c-4a76-be41-78dd6189da8d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:33:33.092544 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.092494    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bdd35ee7-644c-4a76-be41-78dd6189da8d" (UID: "bdd35ee7-644c-4a76-be41-78dd6189da8d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:33:33.092629 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.092549    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bdd35ee7-644c-4a76-be41-78dd6189da8d" (UID: "bdd35ee7-644c-4a76-be41-78dd6189da8d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:33:33.092629 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.092579    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdd35ee7-644c-4a76-be41-78dd6189da8d-kube-api-access-fcjdm" (OuterVolumeSpecName: "kube-api-access-fcjdm") pod "bdd35ee7-644c-4a76-be41-78dd6189da8d" (UID: "bdd35ee7-644c-4a76-be41-78dd6189da8d"). InnerVolumeSpecName "kube-api-access-fcjdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:33:33.134872 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.134823    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:33.192158 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.192126    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd35ee7-644c-4a76-be41-78dd6189da8d-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:33:33.192158 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.192156    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:33:33.192158 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.192166    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdd35ee7-644c-4a76-be41-78dd6189da8d-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:33:33.192379 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.192176    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcjdm\" (UniqueName: \"kubernetes.io/projected/bdd35ee7-644c-4a76-be41-78dd6189da8d-kube-api-access-fcjdm\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:33:33.427168 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.427090    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:33.534055 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.534018    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:33.666192 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.666161    2558 generic.go:356] "Generic (PLEG): container finished" podID="bdd35ee7-644c-4a76-be41-78dd6189da8d" containerID="031ae722e6b57457cd162d0fa810e0127bcf518d71681bdca12fae861f81e05a" exitCode=0
Sep 17 23:33:33.666346 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.666209    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"
Sep 17 23:33:33.666346 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.666250    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99" event={"ID":"bdd35ee7-644c-4a76-be41-78dd6189da8d","Type":"ContainerDied","Data":"031ae722e6b57457cd162d0fa810e0127bcf518d71681bdca12fae861f81e05a"}
Sep 17 23:33:33.666346 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.666302    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99" event={"ID":"bdd35ee7-644c-4a76-be41-78dd6189da8d","Type":"ContainerDied","Data":"fc2bb25082afb8791eb2ba37f17b8a9d1e1a23ebf0f31bd331e9f8e9484d48b8"}
Sep 17 23:33:33.666465 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.666331    2558 scope.go:117] "RemoveContainer" containerID="031ae722e6b57457cd162d0fa810e0127bcf518d71681bdca12fae861f81e05a"
Sep 17 23:33:33.675800 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.675777    2558 scope.go:117] "RemoveContainer" containerID="031ae722e6b57457cd162d0fa810e0127bcf518d71681bdca12fae861f81e05a"
Sep 17 23:33:33.676142 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:33:33.676113    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"031ae722e6b57457cd162d0fa810e0127bcf518d71681bdca12fae861f81e05a\": container with ID starting with 031ae722e6b57457cd162d0fa810e0127bcf518d71681bdca12fae861f81e05a not found: ID does not exist" containerID="031ae722e6b57457cd162d0fa810e0127bcf518d71681bdca12fae861f81e05a"
Sep 17 23:33:33.676247 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.676152    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"031ae722e6b57457cd162d0fa810e0127bcf518d71681bdca12fae861f81e05a"} err="failed to get container status \"031ae722e6b57457cd162d0fa810e0127bcf518d71681bdca12fae861f81e05a\": rpc error: code = NotFound desc = could not find container \"031ae722e6b57457cd162d0fa810e0127bcf518d71681bdca12fae861f81e05a\": container with ID starting with 031ae722e6b57457cd162d0fa810e0127bcf518d71681bdca12fae861f81e05a not found: ID does not exist"
Sep 17 23:33:33.687270 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.687253    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"]
Sep 17 23:33:33.691172 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.691152    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf8993-7cbc46cfbf-nxn99"]
Sep 17 23:33:33.937451 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:33.933643    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdd35ee7-644c-4a76-be41-78dd6189da8d" path="/var/lib/kubelet/pods/bdd35ee7-644c-4a76-be41-78dd6189da8d/volumes"
Sep 17 23:33:34.140834 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:34.140801    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:34.951241 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:34.951213    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:35.725006 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:35.724978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:36.488848 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:36.488818    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:39.101901 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:39.101869    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:39.311177 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:39.311142    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:39.374304 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:39.374234    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:40.020116 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:40.020069    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:40.733135 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:40.733102    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:41.655468 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:41.655438    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:42.290186 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:42.290155    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:43.036646 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:43.036614    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:43.706129 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:43.706099    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:50.253261 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.253227    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"]
Sep 17 23:33:50.253710 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.253420    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw" podUID="e8c59032-063b-40d9-8f0c-cf33117ec60a" containerName="maas-api" containerID="cri-o://6a63aecbc20ef81a5adebb13e4efdc4af49955d25b06d261c9fb931e34b4a78b" gracePeriod=30
Sep 17 23:33:50.288384 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.288360    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:50.386689 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.386651    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:50.482417 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.482389    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:50.503913 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.503855    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:33:50.539699 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.539672    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:50.637185 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.637158    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6qn4\" (UniqueName: \"kubernetes.io/projected/e8c59032-063b-40d9-8f0c-cf33117ec60a-kube-api-access-z6qn4\") pod \"e8c59032-063b-40d9-8f0c-cf33117ec60a\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") "
Sep 17 23:33:50.637376 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.637269    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-maas-api-tls\") pod \"e8c59032-063b-40d9-8f0c-cf33117ec60a\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") "
Sep 17 23:33:50.637424 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.637393    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8c59032-063b-40d9-8f0c-cf33117ec60a-openshift-service-ca\") pod \"e8c59032-063b-40d9-8f0c-cf33117ec60a\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") "
Sep 17 23:33:50.637555 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.637538    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-metrics-certs\") pod \"e8c59032-063b-40d9-8f0c-cf33117ec60a\" (UID: \"e8c59032-063b-40d9-8f0c-cf33117ec60a\") "
Sep 17 23:33:50.637842 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.637811    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8c59032-063b-40d9-8f0c-cf33117ec60a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e8c59032-063b-40d9-8f0c-cf33117ec60a" (UID: "e8c59032-063b-40d9-8f0c-cf33117ec60a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:33:50.639548 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.639500    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c59032-063b-40d9-8f0c-cf33117ec60a-kube-api-access-z6qn4" (OuterVolumeSpecName: "kube-api-access-z6qn4") pod "e8c59032-063b-40d9-8f0c-cf33117ec60a" (UID: "e8c59032-063b-40d9-8f0c-cf33117ec60a"). InnerVolumeSpecName "kube-api-access-z6qn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:33:50.639868 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.639847    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e8c59032-063b-40d9-8f0c-cf33117ec60a" (UID: "e8c59032-063b-40d9-8f0c-cf33117ec60a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:33:50.639944 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.639891    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e8c59032-063b-40d9-8f0c-cf33117ec60a" (UID: "e8c59032-063b-40d9-8f0c-cf33117ec60a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:33:50.738706 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.738676    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:33:50.738706 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.738704    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6qn4\" (UniqueName: \"kubernetes.io/projected/e8c59032-063b-40d9-8f0c-cf33117ec60a-kube-api-access-z6qn4\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:33:50.738875 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.738714    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8c59032-063b-40d9-8f0c-cf33117ec60a-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:33:50.738875 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.738725    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8c59032-063b-40d9-8f0c-cf33117ec60a-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:33:50.756183 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.756110    2558 generic.go:356] "Generic (PLEG): container finished" podID="e8c59032-063b-40d9-8f0c-cf33117ec60a" containerID="6a63aecbc20ef81a5adebb13e4efdc4af49955d25b06d261c9fb931e34b4a78b" exitCode=0
Sep 17 23:33:50.756183 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.756161    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"
Sep 17 23:33:50.756183 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.756160    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw" event={"ID":"e8c59032-063b-40d9-8f0c-cf33117ec60a","Type":"ContainerDied","Data":"6a63aecbc20ef81a5adebb13e4efdc4af49955d25b06d261c9fb931e34b4a78b"}
Sep 17 23:33:50.756380 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.756215    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw" event={"ID":"e8c59032-063b-40d9-8f0c-cf33117ec60a","Type":"ContainerDied","Data":"6dbf93912080ea68f85c37a8b720da5499f7aa87952580c5a8b33f8c2d456db4"}
Sep 17 23:33:50.756380 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.756245    2558 scope.go:117] "RemoveContainer" containerID="6a63aecbc20ef81a5adebb13e4efdc4af49955d25b06d261c9fb931e34b4a78b"
Sep 17 23:33:50.767076 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.767052    2558 scope.go:117] "RemoveContainer" containerID="6a63aecbc20ef81a5adebb13e4efdc4af49955d25b06d261c9fb931e34b4a78b"
Sep 17 23:33:50.767386 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:33:50.767367    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a63aecbc20ef81a5adebb13e4efdc4af49955d25b06d261c9fb931e34b4a78b\": container with ID starting with 6a63aecbc20ef81a5adebb13e4efdc4af49955d25b06d261c9fb931e34b4a78b not found: ID does not exist" containerID="6a63aecbc20ef81a5adebb13e4efdc4af49955d25b06d261c9fb931e34b4a78b"
Sep 17 23:33:50.767444 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.767395    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a63aecbc20ef81a5adebb13e4efdc4af49955d25b06d261c9fb931e34b4a78b"} err="failed to get container status \"6a63aecbc20ef81a5adebb13e4efdc4af49955d25b06d261c9fb931e34b4a78b\": rpc error: code = NotFound desc = could not find container \"6a63aecbc20ef81a5adebb13e4efdc4af49955d25b06d261c9fb931e34b4a78b\": container with ID starting with 6a63aecbc20ef81a5adebb13e4efdc4af49955d25b06d261c9fb931e34b4a78b not found: ID does not exist"
Sep 17 23:33:50.783592 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.783569    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"]
Sep 17 23:33:50.787623 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.787601    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d69250-7b46f5b86c-52dzw"]
Sep 17 23:33:50.850216 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.850189    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:50.951398 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:50.951372    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:51.752300 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:51.752266    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:51.931711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:51.931682    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8c59032-063b-40d9-8f0c-cf33117ec60a" path="/var/lib/kubelet/pods/e8c59032-063b-40d9-8f0c-cf33117ec60a/volumes"
Sep 17 23:33:52.516160 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:52.516126    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:53.350406 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:53.350349    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:56.647103 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:56.647069    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:56.747274 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:56.747236    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:56.918813 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:56.918731    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:57.381464 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:57.381434    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:58.196152 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:58.196119    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:58.925658 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:58.925624    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:33:59.867810 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:33:59.867778    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:00.354424 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:00.354393    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:00.588082 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:00.588050    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:05.441144 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:05.441116    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:07.446554 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:07.446497    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:08.003334 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.003301    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"]
Sep 17 23:34:08.003673 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.003619    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv" podUID="28461752-d1ab-449e-a630-70cd9eb2d00c" containerName="maas-api" containerID="cri-o://65e953e4a4607cc4cfed335b4e270613fcc729821cc79ccae20cb98690969c14" gracePeriod=30
Sep 17 23:34:08.029642 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.029617    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:08.131020 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.130992    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:08.218219 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.218189    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:08.243319 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.243292    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:34:08.280292 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.280232    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:08.382047 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.382017    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lr5wt\" (UniqueName: \"kubernetes.io/projected/28461752-d1ab-449e-a630-70cd9eb2d00c-kube-api-access-lr5wt\") pod \"28461752-d1ab-449e-a630-70cd9eb2d00c\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") "
Sep 17 23:34:08.382285 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.382082    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-maas-api-tls\") pod \"28461752-d1ab-449e-a630-70cd9eb2d00c\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") "
Sep 17 23:34:08.382285 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.382129    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28461752-d1ab-449e-a630-70cd9eb2d00c-openshift-service-ca\") pod \"28461752-d1ab-449e-a630-70cd9eb2d00c\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") "
Sep 17 23:34:08.382285 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.382160    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-metrics-certs\") pod \"28461752-d1ab-449e-a630-70cd9eb2d00c\" (UID: \"28461752-d1ab-449e-a630-70cd9eb2d00c\") "
Sep 17 23:34:08.382959 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.382753    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28461752-d1ab-449e-a630-70cd9eb2d00c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28461752-d1ab-449e-a630-70cd9eb2d00c" (UID: "28461752-d1ab-449e-a630-70cd9eb2d00c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:34:08.384438 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.384413    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28461752-d1ab-449e-a630-70cd9eb2d00c-kube-api-access-lr5wt" (OuterVolumeSpecName: "kube-api-access-lr5wt") pod "28461752-d1ab-449e-a630-70cd9eb2d00c" (UID: "28461752-d1ab-449e-a630-70cd9eb2d00c"). InnerVolumeSpecName "kube-api-access-lr5wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:34:08.384705 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.384677    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "28461752-d1ab-449e-a630-70cd9eb2d00c" (UID: "28461752-d1ab-449e-a630-70cd9eb2d00c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:34:08.384880 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.384862    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28461752-d1ab-449e-a630-70cd9eb2d00c" (UID: "28461752-d1ab-449e-a630-70cd9eb2d00c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:34:08.483537 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.483496    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28461752-d1ab-449e-a630-70cd9eb2d00c-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:34:08.483537 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.483536    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:34:08.483886 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.483547    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lr5wt\" (UniqueName: \"kubernetes.io/projected/28461752-d1ab-449e-a630-70cd9eb2d00c-kube-api-access-lr5wt\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:34:08.483886 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.483555    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28461752-d1ab-449e-a630-70cd9eb2d00c-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:34:08.601407 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.601323    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:08.702158 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.702120    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:08.849870 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.849833    2558 generic.go:356] "Generic (PLEG): container finished" podID="28461752-d1ab-449e-a630-70cd9eb2d00c" containerID="65e953e4a4607cc4cfed335b4e270613fcc729821cc79ccae20cb98690969c14" exitCode=0
Sep 17 23:34:08.850056 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.849878    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"
Sep 17 23:34:08.850056 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.849918    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv" event={"ID":"28461752-d1ab-449e-a630-70cd9eb2d00c","Type":"ContainerDied","Data":"65e953e4a4607cc4cfed335b4e270613fcc729821cc79ccae20cb98690969c14"}
Sep 17 23:34:08.850056 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.849969    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv" event={"ID":"28461752-d1ab-449e-a630-70cd9eb2d00c","Type":"ContainerDied","Data":"7f9d0b640c10426979b2bb50b4b9ec0a4afae81f58c116e4ab8859c11c3d0a77"}
Sep 17 23:34:08.850056 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.849992    2558 scope.go:117] "RemoveContainer" containerID="65e953e4a4607cc4cfed335b4e270613fcc729821cc79ccae20cb98690969c14"
Sep 17 23:34:08.859336 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.859313    2558 scope.go:117] "RemoveContainer" containerID="65e953e4a4607cc4cfed335b4e270613fcc729821cc79ccae20cb98690969c14"
Sep 17 23:34:08.859601 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:34:08.859573    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65e953e4a4607cc4cfed335b4e270613fcc729821cc79ccae20cb98690969c14\": container with ID starting with 65e953e4a4607cc4cfed335b4e270613fcc729821cc79ccae20cb98690969c14 not found: ID does not exist" containerID="65e953e4a4607cc4cfed335b4e270613fcc729821cc79ccae20cb98690969c14"
Sep 17 23:34:08.859730 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.859608    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65e953e4a4607cc4cfed335b4e270613fcc729821cc79ccae20cb98690969c14"} err="failed to get container status \"65e953e4a4607cc4cfed335b4e270613fcc729821cc79ccae20cb98690969c14\": rpc error: code = NotFound desc = could not find container \"65e953e4a4607cc4cfed335b4e270613fcc729821cc79ccae20cb98690969c14\": container with ID starting with 65e953e4a4607cc4cfed335b4e270613fcc729821cc79ccae20cb98690969c14 not found: ID does not exist"
Sep 17 23:34:08.870334 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.870313    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"]
Sep 17 23:34:08.873726 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:08.873707    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fd82a9-5664494474-mklrv"]
Sep 17 23:34:09.451168 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:09.451142    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:09.931302 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:09.931225    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28461752-d1ab-449e-a630-70cd9eb2d00c" path="/var/lib/kubelet/pods/28461752-d1ab-449e-a630-70cd9eb2d00c/volumes"
Sep 17 23:34:10.248460 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:10.248431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:14.431313 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:14.431281    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:14.489142 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:14.489111    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:14.671551 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:14.671509    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"]
Sep 17 23:34:14.671762 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:14.671733    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh" podUID="ad4c855e-b934-466e-8170-7abb14179078" containerName="istio-proxy" containerID="cri-o://7f7459f40d0e9cb9f7799123d6696d2b992d417e3e23485572e82273132ff7c3" gracePeriod=30
Sep 17 23:34:14.695088 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:14.695065    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:19.908188 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.908157    2558 generic.go:356] "Generic (PLEG): container finished" podID="ad4c855e-b934-466e-8170-7abb14179078" containerID="7f7459f40d0e9cb9f7799123d6696d2b992d417e3e23485572e82273132ff7c3" exitCode=0
Sep 17 23:34:19.908488 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.908228    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh" event={"ID":"ad4c855e-b934-466e-8170-7abb14179078","Type":"ContainerDied","Data":"7f7459f40d0e9cb9f7799123d6696d2b992d417e3e23485572e82273132ff7c3"}
Sep 17 23:34:19.908488 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.908320    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh" event={"ID":"ad4c855e-b934-466e-8170-7abb14179078","Type":"ContainerDied","Data":"2aa2444b0ee6e570475b18fbb6f5f137822841c3ddc5109c1b7dd38b90221237"}
Sep 17 23:34:19.908488 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.908338    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2aa2444b0ee6e570475b18fbb6f5f137822841c3ddc5109c1b7dd38b90221237"
Sep 17 23:34:19.908488 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.908352    2558 scope.go:117] "RemoveContainer" containerID="ed528b43a3048e1bb7f78efedaf3034009d956cd3a4bd93d43d89ceed04b6553"
Sep 17 23:34:19.908957 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.908936    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:34:19.972582 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.972560    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-workload-socket\") pod \"ad4c855e-b934-466e-8170-7abb14179078\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") "
Sep 17 23:34:19.972714 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.972587    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad4c855e-b934-466e-8170-7abb14179078-istio-token\") pod \"ad4c855e-b934-466e-8170-7abb14179078\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") "
Sep 17 23:34:19.972714 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.972607    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-workload-certs\") pod \"ad4c855e-b934-466e-8170-7abb14179078\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") "
Sep 17 23:34:19.972714 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.972637    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad4c855e-b934-466e-8170-7abb14179078-istio-podinfo\") pod \"ad4c855e-b934-466e-8170-7abb14179078\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") "
Sep 17 23:34:19.972714 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.972656    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mp45\" (UniqueName: \"kubernetes.io/projected/ad4c855e-b934-466e-8170-7abb14179078-kube-api-access-8mp45\") pod \"ad4c855e-b934-466e-8170-7abb14179078\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") "
Sep 17 23:34:19.972714 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.972686    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-istio-data\") pod \"ad4c855e-b934-466e-8170-7abb14179078\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") "
Sep 17 23:34:19.972976 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.972733    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-istio-envoy\") pod \"ad4c855e-b934-466e-8170-7abb14179078\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") "
Sep 17 23:34:19.972976 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.972759    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-credential-socket\") pod \"ad4c855e-b934-466e-8170-7abb14179078\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") "
Sep 17 23:34:19.972976 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.972791    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad4c855e-b934-466e-8170-7abb14179078-istiod-ca-cert\") pod \"ad4c855e-b934-466e-8170-7abb14179078\" (UID: \"ad4c855e-b934-466e-8170-7abb14179078\") "
Sep 17 23:34:19.972976 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.972882    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ad4c855e-b934-466e-8170-7abb14179078" (UID: "ad4c855e-b934-466e-8170-7abb14179078"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:34:19.972976 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.972909    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ad4c855e-b934-466e-8170-7abb14179078" (UID: "ad4c855e-b934-466e-8170-7abb14179078"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:34:19.973237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.973113    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ad4c855e-b934-466e-8170-7abb14179078" (UID: "ad4c855e-b934-466e-8170-7abb14179078"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:34:19.973296 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.973239    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-istio-data" (OuterVolumeSpecName: "istio-data") pod "ad4c855e-b934-466e-8170-7abb14179078" (UID: "ad4c855e-b934-466e-8170-7abb14179078"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:34:19.973296 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.973264    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-workload-socket\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:34:19.973296 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.973269    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad4c855e-b934-466e-8170-7abb14179078-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ad4c855e-b934-466e-8170-7abb14179078" (UID: "ad4c855e-b934-466e-8170-7abb14179078"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:34:19.973296 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.973286    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-workload-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:34:19.973510 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.973303    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-credential-socket\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:34:19.974776 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.974748    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad4c855e-b934-466e-8170-7abb14179078-istio-token" (OuterVolumeSpecName: "istio-token") pod "ad4c855e-b934-466e-8170-7abb14179078" (UID: "ad4c855e-b934-466e-8170-7abb14179078"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:34:19.974920 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.974902    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ad4c855e-b934-466e-8170-7abb14179078" (UID: "ad4c855e-b934-466e-8170-7abb14179078"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:34:19.975115 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.975095    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad4c855e-b934-466e-8170-7abb14179078-kube-api-access-8mp45" (OuterVolumeSpecName: "kube-api-access-8mp45") pod "ad4c855e-b934-466e-8170-7abb14179078" (UID: "ad4c855e-b934-466e-8170-7abb14179078"). InnerVolumeSpecName "kube-api-access-8mp45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:34:19.975155 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:19.975122    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ad4c855e-b934-466e-8170-7abb14179078-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ad4c855e-b934-466e-8170-7abb14179078" (UID: "ad4c855e-b934-466e-8170-7abb14179078"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:34:20.073952 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:20.073923    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-istio-data\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:34:20.073952 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:20.073949    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad4c855e-b934-466e-8170-7abb14179078-istio-envoy\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:34:20.074124 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:20.073961    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad4c855e-b934-466e-8170-7abb14179078-istiod-ca-cert\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:34:20.074124 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:20.073973    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad4c855e-b934-466e-8170-7abb14179078-istio-token\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:34:20.074124 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:20.073984    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad4c855e-b934-466e-8170-7abb14179078-istio-podinfo\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:34:20.074124 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:20.073995    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mp45\" (UniqueName: \"kubernetes.io/projected/ad4c855e-b934-466e-8170-7abb14179078-kube-api-access-8mp45\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:34:20.914895 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:20.914866    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"
Sep 17 23:34:20.945965 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:20.945933    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"]
Sep 17 23:34:20.949757 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:20.949734    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-fd82a9-openshift-default-64768d66b4-hd6qh"]
Sep 17 23:34:21.930946 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:21.930912    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad4c855e-b934-466e-8170-7abb14179078" path="/var/lib/kubelet/pods/ad4c855e-b934-466e-8170-7abb14179078/volumes"
Sep 17 23:34:53.710553 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.710456    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"]
Sep 17 23:34:53.711153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.710959    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad4c855e-b934-466e-8170-7abb14179078" containerName="istio-proxy"
Sep 17 23:34:53.711153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.710978    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad4c855e-b934-466e-8170-7abb14179078" containerName="istio-proxy"
Sep 17 23:34:53.711153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.710989    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c8c7af9-50bb-4b9c-9421-96e656edcfa1" containerName="maas-api"
Sep 17 23:34:53.711153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.710998    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c8c7af9-50bb-4b9c-9421-96e656edcfa1" containerName="maas-api"
Sep 17 23:34:53.711153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.711022    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28461752-d1ab-449e-a630-70cd9eb2d00c" containerName="maas-api"
Sep 17 23:34:53.711153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.711031    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="28461752-d1ab-449e-a630-70cd9eb2d00c" containerName="maas-api"
Sep 17 23:34:53.711153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.711047    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8c59032-063b-40d9-8f0c-cf33117ec60a" containerName="maas-api"
Sep 17 23:34:53.711153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.711055    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8c59032-063b-40d9-8f0c-cf33117ec60a" containerName="maas-api"
Sep 17 23:34:53.711153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.711065    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdd35ee7-644c-4a76-be41-78dd6189da8d" containerName="maas-api"
Sep 17 23:34:53.711153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.711072    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdd35ee7-644c-4a76-be41-78dd6189da8d" containerName="maas-api"
Sep 17 23:34:53.711153 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.711159    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8c59032-063b-40d9-8f0c-cf33117ec60a" containerName="maas-api"
Sep 17 23:34:53.711587 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.711169    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="28461752-d1ab-449e-a630-70cd9eb2d00c" containerName="maas-api"
Sep 17 23:34:53.711587 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.711180    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad4c855e-b934-466e-8170-7abb14179078" containerName="istio-proxy"
Sep 17 23:34:53.711587 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.711189    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdd35ee7-644c-4a76-be41-78dd6189da8d" containerName="maas-api"
Sep 17 23:34:53.711587 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.711198    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c8c7af9-50bb-4b9c-9421-96e656edcfa1" containerName="maas-api"
Sep 17 23:34:53.711587 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.711204    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad4c855e-b934-466e-8170-7abb14179078" containerName="istio-proxy"
Sep 17 23:34:53.717723 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.717706    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.720028 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.720002    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-d92920\"/\"kube-root-ca.crt\""
Sep 17 23:34:53.720158 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.720007    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-d92920\"/\"openshift-service-ca.crt\""
Sep 17 23:34:53.720158 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.720008    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-d92920\"/\"default-dockercfg-mwr9r\""
Sep 17 23:34:53.720158 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.720071    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-d92920\"/\"rate-test-model-d92920-kserve-self-signed-certs\""
Sep 17 23:34:53.724381 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.724359    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"]
Sep 17 23:34:53.746858 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.746833    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:34:53.822497 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.822463    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-home\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.822497 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.822499    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-dshm\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.822723 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.822549    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2wtb\" (UniqueName: \"kubernetes.io/projected/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-kube-api-access-v2wtb\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.822723 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.822582    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-tmp-dir\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.822723 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.822615    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-model-cache\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.822723 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.822692    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-tls-certs\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.924025 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.924000    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-tls-certs\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.924167 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.924041    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-home\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.924167 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.924062    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-dshm\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.924167 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.924080    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2wtb\" (UniqueName: \"kubernetes.io/projected/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-kube-api-access-v2wtb\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.924167 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.924100    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-tmp-dir\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.924167 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.924136    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-model-cache\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.924457 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.924434    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-home\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.924572 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.924538    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-tmp-dir\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.924572 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.924552    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-model-cache\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.926467 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.926447    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-dshm\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.926585 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.926453    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-tls-certs\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:53.931593 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:53.931571    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2wtb\" (UniqueName: \"kubernetes.io/projected/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-kube-api-access-v2wtb\") pod \"rate-test-model-d92920-kserve-bc9454d7-9qvsq\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") " pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:54.029163 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:54.029114    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:34:54.301162 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:54.301132    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"]
Sep 17 23:34:54.302052 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:34:54.302025    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod244eddda_eb06_4c87_8d8b_bf4b980f3e5e.slice/crio-56c563da9d10d4816ee8063b24df5525e4cdba2ee7885310bb205d2c96492e09 WatchSource:0}: Error finding container 56c563da9d10d4816ee8063b24df5525e4cdba2ee7885310bb205d2c96492e09: Status 404 returned error can't find the container with id 56c563da9d10d4816ee8063b24df5525e4cdba2ee7885310bb205d2c96492e09
Sep 17 23:34:55.092199 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:55.092168    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq" event={"ID":"244eddda-eb06-4c87-8d8b-bf4b980f3e5e","Type":"ContainerStarted","Data":"75883e65e71a91d141d10db1db07e7d59d5b1d7ab9e4d25fa233ce511894958a"}
Sep 17 23:34:55.092199 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:55.092210    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq" event={"ID":"244eddda-eb06-4c87-8d8b-bf4b980f3e5e","Type":"ContainerStarted","Data":"56c563da9d10d4816ee8063b24df5525e4cdba2ee7885310bb205d2c96492e09"}
Sep 17 23:34:55.107380 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:34:55.107335    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq" podStartSLOduration=2.107321901 podStartE2EDuration="2.107321901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:34:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:34:55.106210613 +0000 UTC m=+1719.286849062" watchObservedRunningTime="2026-09-17 23:34:55.107321901 +0000 UTC m=+1719.287960339"
Sep 17 23:35:04.030057 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:04.030023    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:35:04.030057 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:04.030063    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:35:04.042256 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:04.042233    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:35:04.158997 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:04.158964    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:35:18.070502 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:18.070470    2558 scope.go:117] "RemoveContainer" containerID="7f7459f40d0e9cb9f7799123d6696d2b992d417e3e23485572e82273132ff7c3"
Sep 17 23:35:27.163688 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:27.163654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:28.107581 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:28.107552    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"]
Sep 17 23:35:28.128805 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:28.128773    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:28.579639 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:28.579608    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:28.891726 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:28.891646    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:29.151894 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:29.151791    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:31.274064 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:31.274016    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:32.064931 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:32.064893    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:32.163387 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:32.163357    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:32.966207 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:32.966172    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:33.836308 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:33.836274    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"]
Sep 17 23:35:33.836618 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:33.836603    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad4c855e-b934-466e-8170-7abb14179078" containerName="istio-proxy"
Sep 17 23:35:33.836618 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:33.836618    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad4c855e-b934-466e-8170-7abb14179078" containerName="istio-proxy"
Sep 17 23:35:33.841016 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:33.840994    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:33.843753 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:33.843727    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-8f81ee-serving-cert\""
Sep 17 23:35:33.851661 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:33.851607    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"]
Sep 17 23:35:33.877087 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:33.877065    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:33.930489 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:33.930464    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-metrics-certs\") pod \"maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:33.930627 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:33.930498    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:33.930627 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:33.930556    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a62e8c6f-e091-4aaa-b509-a4522a628692-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:33.930740 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:33.930635    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ph89\" (UniqueName: \"kubernetes.io/projected/a62e8c6f-e091-4aaa-b509-a4522a628692-kube-api-access-6ph89\") pod \"maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:33.977542 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:33.977493    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:34.031791 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:34.031761    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ph89\" (UniqueName: \"kubernetes.io/projected/a62e8c6f-e091-4aaa-b509-a4522a628692-kube-api-access-6ph89\") pod \"maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:34.031946 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:34.031812    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-metrics-certs\") pod \"maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:34.031946 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:34.031835    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:34.031946 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:34.031863    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a62e8c6f-e091-4aaa-b509-a4522a628692-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:34.032114 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:35:34.031977    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-serving-cert: secret "maas-api-e2e-ipp-a-8f81ee-serving-cert" not found
Sep 17 23:35:34.032114 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:35:34.032051    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-maas-api-tls podName:a62e8c6f-e091-4aaa-b509-a4522a628692 nodeName:}" failed. No retries permitted until 2026-09-17 23:35:34.532030008 +0000 UTC m=+1758.712668442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-maas-api-tls") pod "maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr" (UID: "a62e8c6f-e091-4aaa-b509-a4522a628692") : secret "maas-api-e2e-ipp-a-8f81ee-serving-cert" not found
Sep 17 23:35:34.032570 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:34.032544    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a62e8c6f-e091-4aaa-b509-a4522a628692-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:34.034217 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:34.034195    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-metrics-certs\") pod \"maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:34.039844 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:34.039824    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ph89\" (UniqueName: \"kubernetes.io/projected/a62e8c6f-e091-4aaa-b509-a4522a628692-kube-api-access-6ph89\") pod \"maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:34.358487 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:34.358459    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:34.462405 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:34.462375    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:34.528592 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:34.528566    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:34.537855 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:34.537836    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:34.540182 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:34.540161    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:34.754615 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:34.754582    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:35.025381 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:35.025356    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"]
Sep 17 23:35:35.027119 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:35:35.027095    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda62e8c6f_e091_4aaa_b509_a4522a628692.slice/crio-7c307f94778cae311b9587799cc42ed11f58c7d3e9e6c7c31e4e97f19a911db8 WatchSource:0}: Error finding container 7c307f94778cae311b9587799cc42ed11f58c7d3e9e6c7c31e4e97f19a911db8: Status 404 returned error can't find the container with id 7c307f94778cae311b9587799cc42ed11f58c7d3e9e6c7c31e4e97f19a911db8
Sep 17 23:35:35.301748 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:35.301679    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr" event={"ID":"a62e8c6f-e091-4aaa-b509-a4522a628692","Type":"ContainerStarted","Data":"7c307f94778cae311b9587799cc42ed11f58c7d3e9e6c7c31e4e97f19a911db8"}
Sep 17 23:35:36.308577 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:36.308537    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr" event={"ID":"a62e8c6f-e091-4aaa-b509-a4522a628692","Type":"ContainerStarted","Data":"29c367a3c755877cbd8d6e4bb97991d0049e00d74fba7489c56fe8db30ffe812"}
Sep 17 23:35:36.308971 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:36.308695    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:36.325388 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:36.325334    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr" podStartSLOduration=2.8015608739999998 podStartE2EDuration="3.325322961s" totalImagesPullingTime="523.762087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:35:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:35:36.323911588 +0000 UTC m=+1760.504550026" watchObservedRunningTime="2026-09-17 23:35:36.325322961 +0000 UTC m=+1760.505961399"
Sep 17 23:35:42.318907 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:42.318878    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:35:43.126259 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.126205    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq" podUID="244eddda-eb06-4c87-8d8b-bf4b980f3e5e" containerName="main" containerID="cri-o://75883e65e71a91d141d10db1db07e7d59d5b1d7ab9e4d25fa233ce511894958a" gracePeriod=45
Sep 17 23:35:43.344701 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.344672    2558 generic.go:356] "Generic (PLEG): container finished" podID="244eddda-eb06-4c87-8d8b-bf4b980f3e5e" containerID="75883e65e71a91d141d10db1db07e7d59d5b1d7ab9e4d25fa233ce511894958a" exitCode=0
Sep 17 23:35:43.345003 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.344730    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq" event={"ID":"244eddda-eb06-4c87-8d8b-bf4b980f3e5e","Type":"ContainerDied","Data":"75883e65e71a91d141d10db1db07e7d59d5b1d7ab9e4d25fa233ce511894958a"}
Sep 17 23:35:43.361082 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.361062    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:35:43.405146 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.405091    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-tls-certs\") pod \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") "
Sep 17 23:35:43.405146 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.405122    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-home\") pod \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") "
Sep 17 23:35:43.405146 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.405139    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2wtb\" (UniqueName: \"kubernetes.io/projected/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-kube-api-access-v2wtb\") pod \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") "
Sep 17 23:35:43.405312 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.405167    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-tmp-dir\") pod \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") "
Sep 17 23:35:43.405312 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.405234    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-dshm\") pod \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") "
Sep 17 23:35:43.405312 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.405257    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-model-cache\") pod \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\" (UID: \"244eddda-eb06-4c87-8d8b-bf4b980f3e5e\") "
Sep 17 23:35:43.405454 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.405405    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-home" (OuterVolumeSpecName: "home") pod "244eddda-eb06-4c87-8d8b-bf4b980f3e5e" (UID: "244eddda-eb06-4c87-8d8b-bf4b980f3e5e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:35:43.405454 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.405440    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "244eddda-eb06-4c87-8d8b-bf4b980f3e5e" (UID: "244eddda-eb06-4c87-8d8b-bf4b980f3e5e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:35:43.405597 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.405573    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-home\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:35:43.405597 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.405590    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-tmp-dir\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:35:43.405685 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.405604    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-model-cache" (OuterVolumeSpecName: "model-cache") pod "244eddda-eb06-4c87-8d8b-bf4b980f3e5e" (UID: "244eddda-eb06-4c87-8d8b-bf4b980f3e5e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:35:43.407247 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.407215    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-dshm" (OuterVolumeSpecName: "dshm") pod "244eddda-eb06-4c87-8d8b-bf4b980f3e5e" (UID: "244eddda-eb06-4c87-8d8b-bf4b980f3e5e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:35:43.407342 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.407265    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-kube-api-access-v2wtb" (OuterVolumeSpecName: "kube-api-access-v2wtb") pod "244eddda-eb06-4c87-8d8b-bf4b980f3e5e" (UID: "244eddda-eb06-4c87-8d8b-bf4b980f3e5e"). InnerVolumeSpecName "kube-api-access-v2wtb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:35:43.407384 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.407336    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "244eddda-eb06-4c87-8d8b-bf4b980f3e5e" (UID: "244eddda-eb06-4c87-8d8b-bf4b980f3e5e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:35:43.506400 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.506379    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-dshm\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:35:43.506400 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.506398    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-model-cache\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:35:43.506559 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.506408    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-tls-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:35:43.506559 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:43.506416    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2wtb\" (UniqueName: \"kubernetes.io/projected/244eddda-eb06-4c87-8d8b-bf4b980f3e5e-kube-api-access-v2wtb\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:35:44.350866 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:44.350785    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"
Sep 17 23:35:44.350866 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:44.350826    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq" event={"ID":"244eddda-eb06-4c87-8d8b-bf4b980f3e5e","Type":"ContainerDied","Data":"56c563da9d10d4816ee8063b24df5525e4cdba2ee7885310bb205d2c96492e09"}
Sep 17 23:35:44.351317 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:44.350870    2558 scope.go:117] "RemoveContainer" containerID="75883e65e71a91d141d10db1db07e7d59d5b1d7ab9e4d25fa233ce511894958a"
Sep 17 23:35:44.366143 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:44.366119    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"]
Sep 17 23:35:44.368307 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:44.368288    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-d92920/rate-test-model-d92920-kserve-bc9454d7-9qvsq"]
Sep 17 23:35:45.931508 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:45.931477    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="244eddda-eb06-4c87-8d8b-bf4b980f3e5e" path="/var/lib/kubelet/pods/244eddda-eb06-4c87-8d8b-bf4b980f3e5e/volumes"
Sep 17 23:35:55.955638 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:55.955594    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:56.615834 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:56.615805    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:57.002389 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:57.002360    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:57.372774 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:57.372691    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:57.680856 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:57.680775    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:57.999102 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:57.999071    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:58.140864 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:58.140835    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:35:58.239731 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:35:58.239695    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:00.652899 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:00.652822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:01.190659 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:01.190621    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:01.280971 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:01.280941    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:01.567855 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:01.567769    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:02.264741 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:02.264701    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:02.921942 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:02.921910    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:03.596915 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.596869    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"]
Sep 17 23:36:03.597383 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.597331    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="244eddda-eb06-4c87-8d8b-bf4b980f3e5e" containerName="main"
Sep 17 23:36:03.597383 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.597349    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="244eddda-eb06-4c87-8d8b-bf4b980f3e5e" containerName="main"
Sep 17 23:36:03.597541 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.597432    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="244eddda-eb06-4c87-8d8b-bf4b980f3e5e" containerName="main"
Sep 17 23:36:03.602191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.602169    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:03.604682 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.604659    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-accfb5-serving-cert\""
Sep 17 23:36:03.611229 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.610883    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"]
Sep 17 23:36:03.638342 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.638310    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:03.669042 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.669008    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4g44\" (UniqueName: \"kubernetes.io/projected/0e42e4f4-ef37-4e6b-bd96-b732207a0789-kube-api-access-d4g44\") pod \"maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:03.669210 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.669046    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-maas-api-tls\") pod \"maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:03.669210 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.669090    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-metrics-certs\") pod \"maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:03.669302 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.669219    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e42e4f4-ef37-4e6b-bd96-b732207a0789-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:03.739507 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.739476    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:03.769824 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.769793    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e42e4f4-ef37-4e6b-bd96-b732207a0789-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:03.769993 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.769833    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4g44\" (UniqueName: \"kubernetes.io/projected/0e42e4f4-ef37-4e6b-bd96-b732207a0789-kube-api-access-d4g44\") pod \"maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:03.769993 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.769852    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-maas-api-tls\") pod \"maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:03.769993 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.769890    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-metrics-certs\") pod \"maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:03.769993 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:36:03.769978    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-serving-cert: secret "maas-api-e2e-ipp-b-accfb5-serving-cert" not found
Sep 17 23:36:03.770250 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:36:03.770042    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-maas-api-tls podName:0e42e4f4-ef37-4e6b-bd96-b732207a0789 nodeName:}" failed. No retries permitted until 2026-09-17 23:36:04.270026322 +0000 UTC m=+1788.450664750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-maas-api-tls") pod "maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g" (UID: "0e42e4f4-ef37-4e6b-bd96-b732207a0789") : secret "maas-api-e2e-ipp-b-accfb5-serving-cert" not found
Sep 17 23:36:03.770562 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.770541    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e42e4f4-ef37-4e6b-bd96-b732207a0789-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:03.772456 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.772435    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-metrics-certs\") pod \"maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:03.778170 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.778146    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4g44\" (UniqueName: \"kubernetes.io/projected/0e42e4f4-ef37-4e6b-bd96-b732207a0789-kube-api-access-d4g44\") pod \"maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:03.991775 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:03.991740    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:04.103057 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:04.103027    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:04.198983 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:04.198950    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:04.275072 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:04.274990    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-maas-api-tls\") pod \"maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:04.277344 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:04.277313    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-maas-api-tls\") pod \"maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:04.515966 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:04.515938    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:04.785255 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:04.785230    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"]
Sep 17 23:36:04.786894 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:36:04.786869    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e42e4f4_ef37_4e6b_bd96_b732207a0789.slice/crio-7811193bba6a02006be066b6bc98cc11d161c35f34406ab01b8f7e04436188de WatchSource:0}: Error finding container 7811193bba6a02006be066b6bc98cc11d161c35f34406ab01b8f7e04436188de: Status 404 returned error can't find the container with id 7811193bba6a02006be066b6bc98cc11d161c35f34406ab01b8f7e04436188de
Sep 17 23:36:05.460111 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:05.460079    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g" event={"ID":"0e42e4f4-ef37-4e6b-bd96-b732207a0789","Type":"ContainerStarted","Data":"068fd6cffaaede839a7437e056ede7d360603502d8173fdad2ec7a4a0d4a4d63"}
Sep 17 23:36:05.460275 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:05.460155    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g" event={"ID":"0e42e4f4-ef37-4e6b-bd96-b732207a0789","Type":"ContainerStarted","Data":"7811193bba6a02006be066b6bc98cc11d161c35f34406ab01b8f7e04436188de"}
Sep 17 23:36:05.460275 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:05.460177    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:05.477829 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:05.477777    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g" podStartSLOduration=2.129322505 podStartE2EDuration="2.477763872s" totalImagesPullingTime="348.441367ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:36:05.4765302 +0000 UTC m=+1789.657168639" watchObservedRunningTime="2026-09-17 23:36:05.477763872 +0000 UTC m=+1789.658402362"
Sep 17 23:36:11.471018 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:11.470988    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:36:25.835830 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:25.835791    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:26.212175 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:26.212145    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:26.566686 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:26.566602    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:27.371020 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:27.370985    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:28.151669 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:28.151638    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:28.940744 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:28.940710    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:30.353264 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:30.353231    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:36:39.656116 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:36:39.656081    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:03.001318 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:03.001282    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:03.315740 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:03.315648    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:03.595308 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:03.595225    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:05.956360 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:05.956319    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:06.653135 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:06.653102    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:06.753321 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:06.753286    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:07.578252 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:07.578219    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:08.362937 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:08.362899    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:09.127717 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.127674    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:09.790166 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.790133    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"]
Sep 17 23:37:09.793750 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.793722    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:09.796099 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.796073    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-e485c4-serving-cert\""
Sep 17 23:37:09.804964 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.804940    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"]
Sep 17 23:37:09.834156 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.834120    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:09.865360 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.865335    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/def53a41-d898-491d-a04b-7e03e1536c7d-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:09.865463 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.865380    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klc7r\" (UniqueName: \"kubernetes.io/projected/def53a41-d898-491d-a04b-7e03e1536c7d-kube-api-access-klc7r\") pod \"maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:09.865463 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.865404    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/def53a41-d898-491d-a04b-7e03e1536c7d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:09.865586 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.865458    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/def53a41-d898-491d-a04b-7e03e1536c7d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:09.937486 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.937464    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:09.966753 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.966726    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/def53a41-d898-491d-a04b-7e03e1536c7d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:09.966860 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.966764    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/def53a41-d898-491d-a04b-7e03e1536c7d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:09.966860 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.966852    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/def53a41-d898-491d-a04b-7e03e1536c7d-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:09.966979 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.966911    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klc7r\" (UniqueName: \"kubernetes.io/projected/def53a41-d898-491d-a04b-7e03e1536c7d-kube-api-access-klc7r\") pod \"maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:09.967442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.967424    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/def53a41-d898-491d-a04b-7e03e1536c7d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:09.969036 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.969017    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/def53a41-d898-491d-a04b-7e03e1536c7d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:09.969111 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.969084    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/def53a41-d898-491d-a04b-7e03e1536c7d-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:09.975561 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:09.975540    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klc7r\" (UniqueName: \"kubernetes.io/projected/def53a41-d898-491d-a04b-7e03e1536c7d-kube-api-access-klc7r\") pod \"maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:10.035644 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:10.035617    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:10.106607 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:10.106553    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:10.292683 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:10.292652    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:10.381541 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:10.380797    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"]
Sep 17 23:37:10.384037 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:10.383991    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:10.384275 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:37:10.384250    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddef53a41_d898_491d_a04b_7e03e1536c7d.slice/crio-1456793e4e18832b8d2fea37ae6e1225e865247f97fc958d811a46a15cf4b6a7 WatchSource:0}: Error finding container 1456793e4e18832b8d2fea37ae6e1225e865247f97fc958d811a46a15cf4b6a7: Status 404 returned error can't find the container with id 1456793e4e18832b8d2fea37ae6e1225e865247f97fc958d811a46a15cf4b6a7
Sep 17 23:37:10.385925 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:10.385909    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:37:10.791009 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:10.790982    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm" event={"ID":"def53a41-d898-491d-a04b-7e03e1536c7d","Type":"ContainerStarted","Data":"1456793e4e18832b8d2fea37ae6e1225e865247f97fc958d811a46a15cf4b6a7"}
Sep 17 23:37:11.797668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:11.797631    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm" event={"ID":"def53a41-d898-491d-a04b-7e03e1536c7d","Type":"ContainerStarted","Data":"35503362829c972eaf47d8c238cc9a1e4e4b95f2ed6add11a7ccbcba05ea0ad2"}
Sep 17 23:37:11.798078 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:11.797745    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:11.817322 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:11.817275    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm" podStartSLOduration=2.438690616 podStartE2EDuration="2.817261582s" totalImagesPullingTime="378.570966ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:37:11.814646081 +0000 UTC m=+1855.995284520" watchObservedRunningTime="2026-09-17 23:37:11.817261582 +0000 UTC m=+1855.997900019"
Sep 17 23:37:16.197199 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:16.197168    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:17.807442 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:17.807406    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:32.944812 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:32.944735    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:33.587205 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:33.587174    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:34.302719 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:34.302682    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:34.403536 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:34.403490    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:35.050581 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:35.050531    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:35.718325 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:35.718290    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:36.548510 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:36.548461    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:36.722320 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:36.722286    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:38.146302 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:38.146269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:38.960379 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:38.960343    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:39.620034 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:39.619997    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:40.267418 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:40.267381    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:41.062834 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:41.062807    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:41.744246 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:41.744205    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:48.456167 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.456138    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"]
Sep 17 23:37:48.456644 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.456379    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm" podUID="def53a41-d898-491d-a04b-7e03e1536c7d" containerName="maas-api" containerID="cri-o://35503362829c972eaf47d8c238cc9a1e4e4b95f2ed6add11a7ccbcba05ea0ad2" gracePeriod=30
Sep 17 23:37:48.484829 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.484798    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:48.553879 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.553849    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:48.614804 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.614773    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:48.683237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.683201    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:48.701405 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.701387    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:48.760492 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.760430    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/def53a41-d898-491d-a04b-7e03e1536c7d-metrics-certs\") pod \"def53a41-d898-491d-a04b-7e03e1536c7d\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") "
Sep 17 23:37:48.760624 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.760500    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/def53a41-d898-491d-a04b-7e03e1536c7d-maas-api-tls\") pod \"def53a41-d898-491d-a04b-7e03e1536c7d\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") "
Sep 17 23:37:48.760695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.760674    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klc7r\" (UniqueName: \"kubernetes.io/projected/def53a41-d898-491d-a04b-7e03e1536c7d-kube-api-access-klc7r\") pod \"def53a41-d898-491d-a04b-7e03e1536c7d\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") "
Sep 17 23:37:48.760771 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.760759    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/def53a41-d898-491d-a04b-7e03e1536c7d-openshift-service-ca\") pod \"def53a41-d898-491d-a04b-7e03e1536c7d\" (UID: \"def53a41-d898-491d-a04b-7e03e1536c7d\") "
Sep 17 23:37:48.761142 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.761119    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/def53a41-d898-491d-a04b-7e03e1536c7d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "def53a41-d898-491d-a04b-7e03e1536c7d" (UID: "def53a41-d898-491d-a04b-7e03e1536c7d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:37:48.762560 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.762535    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/def53a41-d898-491d-a04b-7e03e1536c7d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "def53a41-d898-491d-a04b-7e03e1536c7d" (UID: "def53a41-d898-491d-a04b-7e03e1536c7d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:37:48.762653 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.762578    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/def53a41-d898-491d-a04b-7e03e1536c7d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "def53a41-d898-491d-a04b-7e03e1536c7d" (UID: "def53a41-d898-491d-a04b-7e03e1536c7d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:37:48.762692 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.762660    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/def53a41-d898-491d-a04b-7e03e1536c7d-kube-api-access-klc7r" (OuterVolumeSpecName: "kube-api-access-klc7r") pod "def53a41-d898-491d-a04b-7e03e1536c7d" (UID: "def53a41-d898-491d-a04b-7e03e1536c7d"). InnerVolumeSpecName "kube-api-access-klc7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:37:48.783417 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.783390    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:48.861561 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.861527    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/def53a41-d898-491d-a04b-7e03e1536c7d-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:37:48.861561 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.861560    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/def53a41-d898-491d-a04b-7e03e1536c7d-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:37:48.861839 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.861576    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klc7r\" (UniqueName: \"kubernetes.io/projected/def53a41-d898-491d-a04b-7e03e1536c7d-kube-api-access-klc7r\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:37:48.861839 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.861591    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/def53a41-d898-491d-a04b-7e03e1536c7d-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:37:48.988656 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.988625    2558 generic.go:356] "Generic (PLEG): container finished" podID="def53a41-d898-491d-a04b-7e03e1536c7d" containerID="35503362829c972eaf47d8c238cc9a1e4e4b95f2ed6add11a7ccbcba05ea0ad2" exitCode=0
Sep 17 23:37:48.988831 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.988659    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"
Sep 17 23:37:48.988831 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.988684    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm" event={"ID":"def53a41-d898-491d-a04b-7e03e1536c7d","Type":"ContainerDied","Data":"35503362829c972eaf47d8c238cc9a1e4e4b95f2ed6add11a7ccbcba05ea0ad2"}
Sep 17 23:37:48.988831 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.988734    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm" event={"ID":"def53a41-d898-491d-a04b-7e03e1536c7d","Type":"ContainerDied","Data":"1456793e4e18832b8d2fea37ae6e1225e865247f97fc958d811a46a15cf4b6a7"}
Sep 17 23:37:48.988831 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.988764    2558 scope.go:117] "RemoveContainer" containerID="35503362829c972eaf47d8c238cc9a1e4e4b95f2ed6add11a7ccbcba05ea0ad2"
Sep 17 23:37:48.998931 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.998911    2558 scope.go:117] "RemoveContainer" containerID="35503362829c972eaf47d8c238cc9a1e4e4b95f2ed6add11a7ccbcba05ea0ad2"
Sep 17 23:37:48.999186 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:37:48.999162    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35503362829c972eaf47d8c238cc9a1e4e4b95f2ed6add11a7ccbcba05ea0ad2\": container with ID starting with 35503362829c972eaf47d8c238cc9a1e4e4b95f2ed6add11a7ccbcba05ea0ad2 not found: ID does not exist" containerID="35503362829c972eaf47d8c238cc9a1e4e4b95f2ed6add11a7ccbcba05ea0ad2"
Sep 17 23:37:48.999285 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:48.999191    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35503362829c972eaf47d8c238cc9a1e4e4b95f2ed6add11a7ccbcba05ea0ad2"} err="failed to get container status \"35503362829c972eaf47d8c238cc9a1e4e4b95f2ed6add11a7ccbcba05ea0ad2\": rpc error: code = NotFound desc = could not find container \"35503362829c972eaf47d8c238cc9a1e4e4b95f2ed6add11a7ccbcba05ea0ad2\": container with ID starting with 35503362829c972eaf47d8c238cc9a1e4e4b95f2ed6add11a7ccbcba05ea0ad2 not found: ID does not exist"
Sep 17 23:37:49.010609 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:49.010561    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"]
Sep 17 23:37:49.014664 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:49.014643    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e485c4-54779665bf-n4hwm"]
Sep 17 23:37:49.069197 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:49.069179    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:49.168440 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:49.168416    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:49.843741 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:49.843705    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:49.931906 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:49.931875    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="def53a41-d898-491d-a04b-7e03e1536c7d" path="/var/lib/kubelet/pods/def53a41-d898-491d-a04b-7e03e1536c7d/volumes"
Sep 17 23:37:50.624199 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:50.624168    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:51.459853 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:51.459821    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:52.119656 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:52.119613    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:54.847973 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:54.847940    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:54.903525 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:54.903476    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:55.126721 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:55.126629    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:59.162621 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:59.162562    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:37:59.952123 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:37:59.952062    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:00.626955 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:00.626926    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:01.407263 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:01.407228    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:02.280723 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:02.280660    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:02.943547 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:02.943486    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:03.753903 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:03.753872    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:09.094822 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:09.094784    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:14.680664 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:14.680626    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"]
Sep 17 23:38:14.681061 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:14.680797    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr" podUID="a62e8c6f-e091-4aaa-b509-a4522a628692" containerName="maas-api" containerID="cri-o://29c367a3c755877cbd8d6e4bb97991d0049e00d74fba7489c56fe8db30ffe812" gracePeriod=30
Sep 17 23:38:14.700904 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:14.700857    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:14.799247 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:14.799210    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:14.901145 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:14.901109    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:14.931277 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:14.931220    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:38:14.957581 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:14.957556    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:15.081325 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.081298    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-metrics-certs\") pod \"a62e8c6f-e091-4aaa-b509-a4522a628692\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") "
Sep 17 23:38:15.081551 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.081341    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ph89\" (UniqueName: \"kubernetes.io/projected/a62e8c6f-e091-4aaa-b509-a4522a628692-kube-api-access-6ph89\") pod \"a62e8c6f-e091-4aaa-b509-a4522a628692\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") "
Sep 17 23:38:15.081551 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.081378    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a62e8c6f-e091-4aaa-b509-a4522a628692-openshift-service-ca\") pod \"a62e8c6f-e091-4aaa-b509-a4522a628692\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") "
Sep 17 23:38:15.081551 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.081487    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-maas-api-tls\") pod \"a62e8c6f-e091-4aaa-b509-a4522a628692\" (UID: \"a62e8c6f-e091-4aaa-b509-a4522a628692\") "
Sep 17 23:38:15.081771 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.081747    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a62e8c6f-e091-4aaa-b509-a4522a628692-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a62e8c6f-e091-4aaa-b509-a4522a628692" (UID: "a62e8c6f-e091-4aaa-b509-a4522a628692"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:38:15.081860 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.081841    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a62e8c6f-e091-4aaa-b509-a4522a628692-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:38:15.083929 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.083875    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a62e8c6f-e091-4aaa-b509-a4522a628692-kube-api-access-6ph89" (OuterVolumeSpecName: "kube-api-access-6ph89") pod "a62e8c6f-e091-4aaa-b509-a4522a628692" (UID: "a62e8c6f-e091-4aaa-b509-a4522a628692"). InnerVolumeSpecName "kube-api-access-6ph89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:38:15.084050 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.083934    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a62e8c6f-e091-4aaa-b509-a4522a628692" (UID: "a62e8c6f-e091-4aaa-b509-a4522a628692"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:38:15.084050 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.084022    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a62e8c6f-e091-4aaa-b509-a4522a628692" (UID: "a62e8c6f-e091-4aaa-b509-a4522a628692"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:38:15.125384 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.125354    2558 generic.go:356] "Generic (PLEG): container finished" podID="a62e8c6f-e091-4aaa-b509-a4522a628692" containerID="29c367a3c755877cbd8d6e4bb97991d0049e00d74fba7489c56fe8db30ffe812" exitCode=0
Sep 17 23:38:15.125508 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.125415    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"
Sep 17 23:38:15.125508 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.125437    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr" event={"ID":"a62e8c6f-e091-4aaa-b509-a4522a628692","Type":"ContainerDied","Data":"29c367a3c755877cbd8d6e4bb97991d0049e00d74fba7489c56fe8db30ffe812"}
Sep 17 23:38:15.125508 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.125487    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr" event={"ID":"a62e8c6f-e091-4aaa-b509-a4522a628692","Type":"ContainerDied","Data":"7c307f94778cae311b9587799cc42ed11f58c7d3e9e6c7c31e4e97f19a911db8"}
Sep 17 23:38:15.125508 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.125509    2558 scope.go:117] "RemoveContainer" containerID="29c367a3c755877cbd8d6e4bb97991d0049e00d74fba7489c56fe8db30ffe812"
Sep 17 23:38:15.134264 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.134247    2558 scope.go:117] "RemoveContainer" containerID="29c367a3c755877cbd8d6e4bb97991d0049e00d74fba7489c56fe8db30ffe812"
Sep 17 23:38:15.134576 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:38:15.134557    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29c367a3c755877cbd8d6e4bb97991d0049e00d74fba7489c56fe8db30ffe812\": container with ID starting with 29c367a3c755877cbd8d6e4bb97991d0049e00d74fba7489c56fe8db30ffe812 not found: ID does not exist" containerID="29c367a3c755877cbd8d6e4bb97991d0049e00d74fba7489c56fe8db30ffe812"
Sep 17 23:38:15.134632 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.134585    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c367a3c755877cbd8d6e4bb97991d0049e00d74fba7489c56fe8db30ffe812"} err="failed to get container status \"29c367a3c755877cbd8d6e4bb97991d0049e00d74fba7489c56fe8db30ffe812\": rpc error: code = NotFound desc = could not find container \"29c367a3c755877cbd8d6e4bb97991d0049e00d74fba7489c56fe8db30ffe812\": container with ID starting with 29c367a3c755877cbd8d6e4bb97991d0049e00d74fba7489c56fe8db30ffe812 not found: ID does not exist"
Sep 17 23:38:15.147599 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.147570    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"]
Sep 17 23:38:15.153383 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.153364    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8f81ee-67cd5cfdfb-sf2wr"]
Sep 17 23:38:15.183049 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.182995    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:38:15.183049 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.183028    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a62e8c6f-e091-4aaa-b509-a4522a628692-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:38:15.183049 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.183043    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ph89\" (UniqueName: \"kubernetes.io/projected/a62e8c6f-e091-4aaa-b509-a4522a628692-kube-api-access-6ph89\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:38:15.281014 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.280989    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:15.336741 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.336723    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:15.932129 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:15.932096    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a62e8c6f-e091-4aaa-b509-a4522a628692" path="/var/lib/kubelet/pods/a62e8c6f-e091-4aaa-b509-a4522a628692/volumes"
Sep 17 23:38:16.150786 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:16.150760    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:16.842366 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:16.842334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:17.495923 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:17.495892    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:20.317971 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:20.317940    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:29.521363 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:29.521257    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:56.557975 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:56.557942    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:38:57.194534 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:38:57.194481    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:02.795569 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:02.795466    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:02.868824 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:02.868798    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:03.085358 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:03.085282    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:03.736678 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:03.736643    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:04.523411 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:04.523377    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:05.284339 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:05.284306    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:06.058138 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:06.058108    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:06.974971 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:06.974937    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:07.658767 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:07.658739    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:13.651361 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:13.651327    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:14.193952 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.193921    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"]
Sep 17 23:39:14.194242 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.194215    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g" podUID="0e42e4f4-ef37-4e6b-bd96-b732207a0789" containerName="maas-api" containerID="cri-o://068fd6cffaaede839a7437e056ede7d360603502d8173fdad2ec7a4a0d4a4d63" gracePeriod=30
Sep 17 23:39:14.210873 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.210848    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:14.311560 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.311534    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:14.407312 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.407283    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:14.420649 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.420621    2558 generic.go:356] "Generic (PLEG): container finished" podID="0e42e4f4-ef37-4e6b-bd96-b732207a0789" containerID="068fd6cffaaede839a7437e056ede7d360603502d8173fdad2ec7a4a0d4a4d63" exitCode=0
Sep 17 23:39:14.420755 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.420691    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g" event={"ID":"0e42e4f4-ef37-4e6b-bd96-b732207a0789","Type":"ContainerDied","Data":"068fd6cffaaede839a7437e056ede7d360603502d8173fdad2ec7a4a0d4a4d63"}
Sep 17 23:39:14.420755 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.420740    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g" event={"ID":"0e42e4f4-ef37-4e6b-bd96-b732207a0789","Type":"ContainerDied","Data":"7811193bba6a02006be066b6bc98cc11d161c35f34406ab01b8f7e04436188de"}
Sep 17 23:39:14.420828 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.420758    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7811193bba6a02006be066b6bc98cc11d161c35f34406ab01b8f7e04436188de"
Sep 17 23:39:14.431385 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.431366    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:39:14.479287 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.479230    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:14.535411 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.535385    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4g44\" (UniqueName: \"kubernetes.io/projected/0e42e4f4-ef37-4e6b-bd96-b732207a0789-kube-api-access-d4g44\") pod \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") "
Sep 17 23:39:14.535531 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.535439    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-maas-api-tls\") pod \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") "
Sep 17 23:39:14.535531 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.535473    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-metrics-certs\") pod \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") "
Sep 17 23:39:14.535531 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.535501    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e42e4f4-ef37-4e6b-bd96-b732207a0789-openshift-service-ca\") pod \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\" (UID: \"0e42e4f4-ef37-4e6b-bd96-b732207a0789\") "
Sep 17 23:39:14.535916 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.535886    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e42e4f4-ef37-4e6b-bd96-b732207a0789-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0e42e4f4-ef37-4e6b-bd96-b732207a0789" (UID: "0e42e4f4-ef37-4e6b-bd96-b732207a0789"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:39:14.537552 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.537487    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0e42e4f4-ef37-4e6b-bd96-b732207a0789" (UID: "0e42e4f4-ef37-4e6b-bd96-b732207a0789"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:39:14.537642 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.537558    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0e42e4f4-ef37-4e6b-bd96-b732207a0789" (UID: "0e42e4f4-ef37-4e6b-bd96-b732207a0789"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:39:14.537686 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.537651    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e42e4f4-ef37-4e6b-bd96-b732207a0789-kube-api-access-d4g44" (OuterVolumeSpecName: "kube-api-access-d4g44") pod "0e42e4f4-ef37-4e6b-bd96-b732207a0789" (UID: "0e42e4f4-ef37-4e6b-bd96-b732207a0789"). InnerVolumeSpecName "kube-api-access-d4g44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:39:14.636299 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.636266    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:39:14.636299 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.636295    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e42e4f4-ef37-4e6b-bd96-b732207a0789-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:39:14.636299 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.636306    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4g44\" (UniqueName: \"kubernetes.io/projected/0e42e4f4-ef37-4e6b-bd96-b732207a0789-kube-api-access-d4g44\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:39:14.636588 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.636315    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e42e4f4-ef37-4e6b-bd96-b732207a0789-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:39:14.812745 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.812662    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:14.913354 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:14.913324    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:15.425818 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:15.425790    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"
Sep 17 23:39:15.450577 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:15.450548    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"]
Sep 17 23:39:15.452557 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:15.452536    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-accfb5-c8d87cf6-dqc9g"]
Sep 17 23:39:15.505995 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:15.505973    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:15.935685 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:15.935649    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e42e4f4-ef37-4e6b-bd96-b732207a0789" path="/var/lib/kubelet/pods/0e42e4f4-ef37-4e6b-bd96-b732207a0789/volumes"
Sep 17 23:39:16.178733 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:16.178704    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:20.518032 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:20.517999    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:20.619654 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:20.619622    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:20.797542 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:20.797449    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:21.906233 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:21.906184    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:22.630766 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:22.630683    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:23.294806 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:23.294765    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:31.568182 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.568148    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"]
Sep 17 23:39:31.568817 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.568378    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph" podUID="fea943a5-44f2-4f4c-9270-6b5c0b716731" containerName="maas-api" containerID="cri-o://db6fc5204d51c3470649dc3ffc437dddf635afc571c5fbca477394aee92c72b0" gracePeriod=30
Sep 17 23:39:31.601040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.600998    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:31.666935 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.666899    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:31.728820 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.728784    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:31.796271 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.796240    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:31.810251 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.810230    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:39:31.851040 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.850975    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:31.871499 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.871473    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4526w\" (UniqueName: \"kubernetes.io/projected/fea943a5-44f2-4f4c-9270-6b5c0b716731-kube-api-access-4526w\") pod \"fea943a5-44f2-4f4c-9270-6b5c0b716731\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") "
Sep 17 23:39:31.871636 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.871536    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-maas-api-tls\") pod \"fea943a5-44f2-4f4c-9270-6b5c0b716731\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") "
Sep 17 23:39:31.871636 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.871580    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fea943a5-44f2-4f4c-9270-6b5c0b716731-openshift-service-ca\") pod \"fea943a5-44f2-4f4c-9270-6b5c0b716731\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") "
Sep 17 23:39:31.871636 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.871605    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-metrics-certs\") pod \"fea943a5-44f2-4f4c-9270-6b5c0b716731\" (UID: \"fea943a5-44f2-4f4c-9270-6b5c0b716731\") "
Sep 17 23:39:31.871983 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.871956    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fea943a5-44f2-4f4c-9270-6b5c0b716731-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fea943a5-44f2-4f4c-9270-6b5c0b716731" (UID: "fea943a5-44f2-4f4c-9270-6b5c0b716731"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:39:31.873632 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.873600    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea943a5-44f2-4f4c-9270-6b5c0b716731-kube-api-access-4526w" (OuterVolumeSpecName: "kube-api-access-4526w") pod "fea943a5-44f2-4f4c-9270-6b5c0b716731" (UID: "fea943a5-44f2-4f4c-9270-6b5c0b716731"). InnerVolumeSpecName "kube-api-access-4526w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:39:31.873757 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.873735    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fea943a5-44f2-4f4c-9270-6b5c0b716731" (UID: "fea943a5-44f2-4f4c-9270-6b5c0b716731"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:39:31.873826 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.873761    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fea943a5-44f2-4f4c-9270-6b5c0b716731" (UID: "fea943a5-44f2-4f4c-9270-6b5c0b716731"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:39:31.972376 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.972337    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4526w\" (UniqueName: \"kubernetes.io/projected/fea943a5-44f2-4f4c-9270-6b5c0b716731-kube-api-access-4526w\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:39:31.972376 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.972373    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:39:31.972586 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.972387    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fea943a5-44f2-4f4c-9270-6b5c0b716731-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:39:31.972586 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:31.972399    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea943a5-44f2-4f4c-9270-6b5c0b716731-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:39:32.171783 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:32.171703    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:32.271504 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:32.271477    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:32.512571 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:32.512544    2558 generic.go:356] "Generic (PLEG): container finished" podID="fea943a5-44f2-4f4c-9270-6b5c0b716731" containerID="db6fc5204d51c3470649dc3ffc437dddf635afc571c5fbca477394aee92c72b0" exitCode=0
Sep 17 23:39:32.512727 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:32.512639    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph" event={"ID":"fea943a5-44f2-4f4c-9270-6b5c0b716731","Type":"ContainerDied","Data":"db6fc5204d51c3470649dc3ffc437dddf635afc571c5fbca477394aee92c72b0"}
Sep 17 23:39:32.512727 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:32.512643    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"
Sep 17 23:39:32.512727 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:32.512678    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph" event={"ID":"fea943a5-44f2-4f4c-9270-6b5c0b716731","Type":"ContainerDied","Data":"dcee09589943e460b1d6f840c87aab22e34d860d35d488d01b84b6f9ad1201b2"}
Sep 17 23:39:32.512727 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:32.512703    2558 scope.go:117] "RemoveContainer" containerID="db6fc5204d51c3470649dc3ffc437dddf635afc571c5fbca477394aee92c72b0"
Sep 17 23:39:32.521898 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:32.521871    2558 scope.go:117] "RemoveContainer" containerID="db6fc5204d51c3470649dc3ffc437dddf635afc571c5fbca477394aee92c72b0"
Sep 17 23:39:32.522140 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:39:32.522121    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db6fc5204d51c3470649dc3ffc437dddf635afc571c5fbca477394aee92c72b0\": container with ID starting with db6fc5204d51c3470649dc3ffc437dddf635afc571c5fbca477394aee92c72b0 not found: ID does not exist" containerID="db6fc5204d51c3470649dc3ffc437dddf635afc571c5fbca477394aee92c72b0"
Sep 17 23:39:32.522195 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:32.522147    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db6fc5204d51c3470649dc3ffc437dddf635afc571c5fbca477394aee92c72b0"} err="failed to get container status \"db6fc5204d51c3470649dc3ffc437dddf635afc571c5fbca477394aee92c72b0\": rpc error: code = NotFound desc = could not find container \"db6fc5204d51c3470649dc3ffc437dddf635afc571c5fbca477394aee92c72b0\": container with ID starting with db6fc5204d51c3470649dc3ffc437dddf635afc571c5fbca477394aee92c72b0 not found: ID does not exist"
Sep 17 23:39:32.528610 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:32.528588    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"]
Sep 17 23:39:32.534668 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:32.534644    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d90df4-6d876bc8f5-c4kph"]
Sep 17 23:39:33.003803 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:33.003774    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:33.931335 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:33.931298    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fea943a5-44f2-4f4c-9270-6b5c0b716731" path="/var/lib/kubelet/pods/fea943a5-44f2-4f4c-9270-6b5c0b716731/volumes"
Sep 17 23:39:37.987129 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:37.987100    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:38.121883 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:38.121852    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:38.298841 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:38.298772    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:39.363063 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:39.363027    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:40.053497 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:40.053468    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:46.353695 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:46.353663    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:49.203749 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.203719    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"]
Sep 17 23:39:49.204209 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.203930    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5" podUID="ffcd9610-6381-4a07-ad1a-086b475ab60e" containerName="maas-api" containerID="cri-o://c0c5b81352a62e427d2ea20c8c78b568684c5d2a1535145a7fc9a7f9b18a4ef9" gracePeriod=30
Sep 17 23:39:49.233488 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.233463    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:49.332898 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.332873    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:49.422117 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.422090    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:49.448880 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.448855    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:39:49.476140 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.476083    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:49.494237 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.494210    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcd9610-6381-4a07-ad1a-086b475ab60e-openshift-service-ca\") pod \"ffcd9610-6381-4a07-ad1a-086b475ab60e\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") "
Sep 17 23:39:49.494349 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.494275    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-metrics-certs\") pod \"ffcd9610-6381-4a07-ad1a-086b475ab60e\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") "
Sep 17 23:39:49.494349 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.494336    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-maas-api-tls\") pod \"ffcd9610-6381-4a07-ad1a-086b475ab60e\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") "
Sep 17 23:39:49.494469 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.494360    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66qvl\" (UniqueName: \"kubernetes.io/projected/ffcd9610-6381-4a07-ad1a-086b475ab60e-kube-api-access-66qvl\") pod \"ffcd9610-6381-4a07-ad1a-086b475ab60e\" (UID: \"ffcd9610-6381-4a07-ad1a-086b475ab60e\") "
Sep 17 23:39:49.494636 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.494610    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffcd9610-6381-4a07-ad1a-086b475ab60e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ffcd9610-6381-4a07-ad1a-086b475ab60e" (UID: "ffcd9610-6381-4a07-ad1a-086b475ab60e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:39:49.496386 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.496366    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffcd9610-6381-4a07-ad1a-086b475ab60e-kube-api-access-66qvl" (OuterVolumeSpecName: "kube-api-access-66qvl") pod "ffcd9610-6381-4a07-ad1a-086b475ab60e" (UID: "ffcd9610-6381-4a07-ad1a-086b475ab60e"). InnerVolumeSpecName "kube-api-access-66qvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:39:49.496386 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.496373    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ffcd9610-6381-4a07-ad1a-086b475ab60e" (UID: "ffcd9610-6381-4a07-ad1a-086b475ab60e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:39:49.496498 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.496429    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ffcd9610-6381-4a07-ad1a-086b475ab60e" (UID: "ffcd9610-6381-4a07-ad1a-086b475ab60e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:39:49.595443 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.595412    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcd9610-6381-4a07-ad1a-086b475ab60e-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:39:49.595443 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.595444    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:39:49.595659 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.595456    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcd9610-6381-4a07-ad1a-086b475ab60e-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:39:49.595659 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.595468    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66qvl\" (UniqueName: \"kubernetes.io/projected/ffcd9610-6381-4a07-ad1a-086b475ab60e-kube-api-access-66qvl\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:39:49.602917 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.602883    2558 generic.go:356] "Generic (PLEG): container finished" podID="ffcd9610-6381-4a07-ad1a-086b475ab60e" containerID="c0c5b81352a62e427d2ea20c8c78b568684c5d2a1535145a7fc9a7f9b18a4ef9" exitCode=0
Sep 17 23:39:49.603034 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.602928    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"
Sep 17 23:39:49.603034 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.602930    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5" event={"ID":"ffcd9610-6381-4a07-ad1a-086b475ab60e","Type":"ContainerDied","Data":"c0c5b81352a62e427d2ea20c8c78b568684c5d2a1535145a7fc9a7f9b18a4ef9"}
Sep 17 23:39:49.603034 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.602994    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5" event={"ID":"ffcd9610-6381-4a07-ad1a-086b475ab60e","Type":"ContainerDied","Data":"75639c7bfa404e30b9cbe8f229905305a0b19100ccf204d4858511c46938d871"}
Sep 17 23:39:49.603034 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.603025    2558 scope.go:117] "RemoveContainer" containerID="c0c5b81352a62e427d2ea20c8c78b568684c5d2a1535145a7fc9a7f9b18a4ef9"
Sep 17 23:39:49.612539 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.612497    2558 scope.go:117] "RemoveContainer" containerID="c0c5b81352a62e427d2ea20c8c78b568684c5d2a1535145a7fc9a7f9b18a4ef9"
Sep 17 23:39:49.612781 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:39:49.612765    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0c5b81352a62e427d2ea20c8c78b568684c5d2a1535145a7fc9a7f9b18a4ef9\": container with ID starting with c0c5b81352a62e427d2ea20c8c78b568684c5d2a1535145a7fc9a7f9b18a4ef9 not found: ID does not exist" containerID="c0c5b81352a62e427d2ea20c8c78b568684c5d2a1535145a7fc9a7f9b18a4ef9"
Sep 17 23:39:49.612829 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.612788    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0c5b81352a62e427d2ea20c8c78b568684c5d2a1535145a7fc9a7f9b18a4ef9"} err="failed to get container status \"c0c5b81352a62e427d2ea20c8c78b568684c5d2a1535145a7fc9a7f9b18a4ef9\": rpc error: code = NotFound desc = could not find container \"c0c5b81352a62e427d2ea20c8c78b568684c5d2a1535145a7fc9a7f9b18a4ef9\": container with ID starting with c0c5b81352a62e427d2ea20c8c78b568684c5d2a1535145a7fc9a7f9b18a4ef9 not found: ID does not exist"
Sep 17 23:39:49.624152 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.624129    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"]
Sep 17 23:39:49.627603 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.627578    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-d92920-79b9564fd7-5cpp5"]
Sep 17 23:39:49.822475 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.822388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:49.884275 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.884242    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:49.933571 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:49.933540    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffcd9610-6381-4a07-ad1a-086b475ab60e" path="/var/lib/kubelet/pods/ffcd9610-6381-4a07-ad1a-086b475ab60e/volumes"
Sep 17 23:39:55.841917 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:55.841885    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:55.942735 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:55.942704    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:56.042802 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:56.042775    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:39:56.214729 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:39:56.214701    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:40:47.836437 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:40:47.836362    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:40:47.937654 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:40:47.937622    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:41:22.374274 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:41:22.374239    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:41:23.912238 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:41:23.912203    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:41:29.496845 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:41:29.496811    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:42:18.153383 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:42:18.153352    2558 scope.go:117] "RemoveContainer" containerID="068fd6cffaaede839a7437e056ede7d360603502d8173fdad2ec7a4a0d4a4d63"
Sep 17 23:43:53.801598 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:43:53.801497    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:43:54.943740 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:43:54.943702    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:44:03.909175 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:44:03.909144    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:44:04.010651 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:44:04.010621    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:44:31.754371 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:44:31.754342    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:44:48.066058 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:44:48.066020    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:19.871847 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:19.871813    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:21.438676 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:21.438648    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:40.572624 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.572589    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"]
Sep 17 23:45:40.573051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.572911    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="def53a41-d898-491d-a04b-7e03e1536c7d" containerName="maas-api"
Sep 17 23:45:40.573051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.572926    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="def53a41-d898-491d-a04b-7e03e1536c7d" containerName="maas-api"
Sep 17 23:45:40.573051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.572940    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a62e8c6f-e091-4aaa-b509-a4522a628692" containerName="maas-api"
Sep 17 23:45:40.573051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.572946    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a62e8c6f-e091-4aaa-b509-a4522a628692" containerName="maas-api"
Sep 17 23:45:40.573051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.572959    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffcd9610-6381-4a07-ad1a-086b475ab60e" containerName="maas-api"
Sep 17 23:45:40.573051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.572964    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffcd9610-6381-4a07-ad1a-086b475ab60e" containerName="maas-api"
Sep 17 23:45:40.573051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.572972    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fea943a5-44f2-4f4c-9270-6b5c0b716731" containerName="maas-api"
Sep 17 23:45:40.573051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.572977    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fea943a5-44f2-4f4c-9270-6b5c0b716731" containerName="maas-api"
Sep 17 23:45:40.573051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.572984    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e42e4f4-ef37-4e6b-bd96-b732207a0789" containerName="maas-api"
Sep 17 23:45:40.573051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.572992    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e42e4f4-ef37-4e6b-bd96-b732207a0789" containerName="maas-api"
Sep 17 23:45:40.573051 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.573056    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffcd9610-6381-4a07-ad1a-086b475ab60e" containerName="maas-api"
Sep 17 23:45:40.573388 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.573062    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a62e8c6f-e091-4aaa-b509-a4522a628692" containerName="maas-api"
Sep 17 23:45:40.573388 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.573074    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="def53a41-d898-491d-a04b-7e03e1536c7d" containerName="maas-api"
Sep 17 23:45:40.573388 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.573083    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="fea943a5-44f2-4f4c-9270-6b5c0b716731" containerName="maas-api"
Sep 17 23:45:40.573388 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.573089    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e42e4f4-ef37-4e6b-bd96-b732207a0789" containerName="maas-api"
Sep 17 23:45:40.575980 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.575965    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:40.586714 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.586688    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"]
Sep 17 23:45:40.619223 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.619197    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd76564-bf06-40eb-a647-adbad9f1d95c-maas-api-tls\") pod \"maas-api-59cb547587-9tfbj\" (UID: \"dcd76564-bf06-40eb-a647-adbad9f1d95c\") " pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:40.619366 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.619241    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wj9f\" (UniqueName: \"kubernetes.io/projected/dcd76564-bf06-40eb-a647-adbad9f1d95c-kube-api-access-7wj9f\") pod \"maas-api-59cb547587-9tfbj\" (UID: \"dcd76564-bf06-40eb-a647-adbad9f1d95c\") " pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:40.619366 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.619290    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcd76564-bf06-40eb-a647-adbad9f1d95c-metrics-certs\") pod \"maas-api-59cb547587-9tfbj\" (UID: \"dcd76564-bf06-40eb-a647-adbad9f1d95c\") " pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:40.619366 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.619320    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcd76564-bf06-40eb-a647-adbad9f1d95c-openshift-service-ca\") pod \"maas-api-59cb547587-9tfbj\" (UID: \"dcd76564-bf06-40eb-a647-adbad9f1d95c\") " pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:40.720129 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.720102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd76564-bf06-40eb-a647-adbad9f1d95c-maas-api-tls\") pod \"maas-api-59cb547587-9tfbj\" (UID: \"dcd76564-bf06-40eb-a647-adbad9f1d95c\") " pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:40.720268 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.720147    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wj9f\" (UniqueName: \"kubernetes.io/projected/dcd76564-bf06-40eb-a647-adbad9f1d95c-kube-api-access-7wj9f\") pod \"maas-api-59cb547587-9tfbj\" (UID: \"dcd76564-bf06-40eb-a647-adbad9f1d95c\") " pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:40.720340 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.720269    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcd76564-bf06-40eb-a647-adbad9f1d95c-metrics-certs\") pod \"maas-api-59cb547587-9tfbj\" (UID: \"dcd76564-bf06-40eb-a647-adbad9f1d95c\") " pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:40.720340 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.720315    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcd76564-bf06-40eb-a647-adbad9f1d95c-openshift-service-ca\") pod \"maas-api-59cb547587-9tfbj\" (UID: \"dcd76564-bf06-40eb-a647-adbad9f1d95c\") " pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:40.720933 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.720905    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcd76564-bf06-40eb-a647-adbad9f1d95c-openshift-service-ca\") pod \"maas-api-59cb547587-9tfbj\" (UID: \"dcd76564-bf06-40eb-a647-adbad9f1d95c\") " pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:40.722742 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.722717    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcd76564-bf06-40eb-a647-adbad9f1d95c-metrics-certs\") pod \"maas-api-59cb547587-9tfbj\" (UID: \"dcd76564-bf06-40eb-a647-adbad9f1d95c\") " pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:40.722842 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.722717    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcd76564-bf06-40eb-a647-adbad9f1d95c-maas-api-tls\") pod \"maas-api-59cb547587-9tfbj\" (UID: \"dcd76564-bf06-40eb-a647-adbad9f1d95c\") " pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:40.726797 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.726773    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wj9f\" (UniqueName: \"kubernetes.io/projected/dcd76564-bf06-40eb-a647-adbad9f1d95c-kube-api-access-7wj9f\") pod \"maas-api-59cb547587-9tfbj\" (UID: \"dcd76564-bf06-40eb-a647-adbad9f1d95c\") " pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:40.888080 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:40.888023    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:41.159383 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:41.159356    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"]
Sep 17 23:45:41.161262 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:45:41.161233    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd76564_bf06_40eb_a647_adbad9f1d95c.slice/crio-227a10402a86834676d60d241e7072178d88351b0496a041c5a34b31bdbb5d15 WatchSource:0}: Error finding container 227a10402a86834676d60d241e7072178d88351b0496a041c5a34b31bdbb5d15: Status 404 returned error can't find the container with id 227a10402a86834676d60d241e7072178d88351b0496a041c5a34b31bdbb5d15
Sep 17 23:45:41.162627 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:41.162605    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:45:41.315044 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:41.315017    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj" event={"ID":"dcd76564-bf06-40eb-a647-adbad9f1d95c","Type":"ContainerStarted","Data":"227a10402a86834676d60d241e7072178d88351b0496a041c5a34b31bdbb5d15"}
Sep 17 23:45:42.321016 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:42.320981    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj" event={"ID":"dcd76564-bf06-40eb-a647-adbad9f1d95c","Type":"ContainerStarted","Data":"2f347d21786f4fa35c361894d196178747bcd8ee7574811cd31387a7086905e2"}
Sep 17 23:45:42.321388 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:42.321082    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:42.339603 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:42.339561    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj" podStartSLOduration=1.934844207 podStartE2EDuration="2.339547884s" totalImagesPullingTime="404.703677ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:45:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:45:42.33902297 +0000 UTC m=+2366.519661432" watchObservedRunningTime="2026-09-17 23:45:42.339547884 +0000 UTC m=+2366.520186322"
Sep 17 23:45:48.333191 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:48.333159    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59cb547587-9tfbj"
Sep 17 23:45:55.712527 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:55.712477    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"]
Sep 17 23:45:55.712998 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:55.712716    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl" podUID="e7d6c6f6-3464-4212-8a1d-08cdb3d889ef" containerName="maas-api" containerID="cri-o://369bfabd73d0c03c7e93a5c9af006d8bf378c4882f546a831505e60b2fbd0194" gracePeriod=30
Sep 17 23:45:55.931459 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:55.931423    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl" podUID="e7d6c6f6-3464-4212-8a1d-08cdb3d889ef" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.32:8443/health\": dial tcp 10.134.0.32:8443: connect: connection refused"
Sep 17 23:45:56.959711 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:56.959687    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:45:57.041220 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.041192    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-openshift-service-ca\") pod \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") "
Sep 17 23:45:57.041368 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.041234    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-maas-api-tls\") pod \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") "
Sep 17 23:45:57.041368 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.041269    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thlhg\" (UniqueName: \"kubernetes.io/projected/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-kube-api-access-thlhg\") pod \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") "
Sep 17 23:45:57.041368 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.041285    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-metrics-certs\") pod \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\" (UID: \"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef\") "
Sep 17 23:45:57.041833 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.041612    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e7d6c6f6-3464-4212-8a1d-08cdb3d889ef" (UID: "e7d6c6f6-3464-4212-8a1d-08cdb3d889ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:45:57.043441 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.043418    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-kube-api-access-thlhg" (OuterVolumeSpecName: "kube-api-access-thlhg") pod "e7d6c6f6-3464-4212-8a1d-08cdb3d889ef" (UID: "e7d6c6f6-3464-4212-8a1d-08cdb3d889ef"). InnerVolumeSpecName "kube-api-access-thlhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:45:57.043838 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.043814    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e7d6c6f6-3464-4212-8a1d-08cdb3d889ef" (UID: "e7d6c6f6-3464-4212-8a1d-08cdb3d889ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:45:57.043893 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.043868    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e7d6c6f6-3464-4212-8a1d-08cdb3d889ef" (UID: "e7d6c6f6-3464-4212-8a1d-08cdb3d889ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:45:57.141920 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.141868    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thlhg\" (UniqueName: \"kubernetes.io/projected/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-kube-api-access-thlhg\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:45:57.141920 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.141888    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-metrics-certs\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:45:57.141920 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.141898    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-openshift-service-ca\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:45:57.141920 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.141907    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef-maas-api-tls\") on node \"ip-10-0-137-236.ec2.internal\" DevicePath \"\""
Sep 17 23:45:57.396022 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.395954    2558 generic.go:356] "Generic (PLEG): container finished" podID="e7d6c6f6-3464-4212-8a1d-08cdb3d889ef" containerID="369bfabd73d0c03c7e93a5c9af006d8bf378c4882f546a831505e60b2fbd0194" exitCode=0
Sep 17 23:45:57.396022 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.395995    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"
Sep 17 23:45:57.396185 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.396043    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl" event={"ID":"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef","Type":"ContainerDied","Data":"369bfabd73d0c03c7e93a5c9af006d8bf378c4882f546a831505e60b2fbd0194"}
Sep 17 23:45:57.396185 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.396097    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl" event={"ID":"e7d6c6f6-3464-4212-8a1d-08cdb3d889ef","Type":"ContainerDied","Data":"a5f406fefa534e0e297a103c7217da129141479d583f44c2966967de86653a7d"}
Sep 17 23:45:57.396185 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.396118    2558 scope.go:117] "RemoveContainer" containerID="369bfabd73d0c03c7e93a5c9af006d8bf378c4882f546a831505e60b2fbd0194"
Sep 17 23:45:57.405155 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.405140    2558 scope.go:117] "RemoveContainer" containerID="369bfabd73d0c03c7e93a5c9af006d8bf378c4882f546a831505e60b2fbd0194"
Sep 17 23:45:57.405394 ip-10-0-137-236 kubenswrapper[2558]: E0917 23:45:57.405377    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"369bfabd73d0c03c7e93a5c9af006d8bf378c4882f546a831505e60b2fbd0194\": container with ID starting with 369bfabd73d0c03c7e93a5c9af006d8bf378c4882f546a831505e60b2fbd0194 not found: ID does not exist" containerID="369bfabd73d0c03c7e93a5c9af006d8bf378c4882f546a831505e60b2fbd0194"
Sep 17 23:45:57.405464 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.405405    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"369bfabd73d0c03c7e93a5c9af006d8bf378c4882f546a831505e60b2fbd0194"} err="failed to get container status \"369bfabd73d0c03c7e93a5c9af006d8bf378c4882f546a831505e60b2fbd0194\": rpc error: code = NotFound desc = could not find container \"369bfabd73d0c03c7e93a5c9af006d8bf378c4882f546a831505e60b2fbd0194\": container with ID starting with 369bfabd73d0c03c7e93a5c9af006d8bf378c4882f546a831505e60b2fbd0194 not found: ID does not exist"
Sep 17 23:45:57.416737 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.416713    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"]
Sep 17 23:45:57.418733 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.418713    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6678c8db5f-2l4wl"]
Sep 17 23:45:57.875121 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.875070    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:45:57.932294 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.932258    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7d6c6f6-3464-4212-8a1d-08cdb3d889ef" path="/var/lib/kubelet/pods/e7d6c6f6-3464-4212-8a1d-08cdb3d889ef/volumes"
Sep 17 23:45:57.974398 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:45:57.974374    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:46:01.858405 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:01.858373    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:46:16.721418 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:16.721389    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-r9ddr_85bee3c1-36a2-486a-bb72-b48cef7565c9/manager/0.log"
Sep 17 23:46:16.981506 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:16.981435    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-747fdf4bf5-t85kq_253192e6-6970-47fe-bba9-ab238e390a33/copy-manifests/0.log"
Sep 17 23:46:16.987469 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:16.987453    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-747fdf4bf5-t85kq_253192e6-6970-47fe-bba9-ab238e390a33/manager/0.log"
Sep 17 23:46:17.470958 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:17.470927    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-59cb547587-9tfbj_dcd76564-bf06-40eb-a647-adbad9f1d95c/maas-api/0.log"
Sep 17 23:46:19.799308 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:19.799277    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx_17d06310-4705-4d83-a87d-c63d35283c59/istio-proxy/0.log"
Sep 17 23:46:19.919531 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:19.919493    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cz26c_f7ff21c7-43e7-4085-855d-03e61bd04eb5/discovery/0.log"
Sep 17 23:46:20.992058 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:20.992027    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-lrht2_6bebec92-9bb5-42b1-b380-3b52a133ce6e/main/0.log"
Sep 17 23:46:21.344015 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:21.343945    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-8sp7c_6d930dce-88ac-4544-b825-71427c0e05c8/main/0.log"
Sep 17 23:46:21.464631 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:21.464612    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5r2q5_3ccfbb47-c4a8-4ce0-90ad-496e117682cd/main/0.log"
Sep 17 23:46:27.836566 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:27.836532    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:46:27.937875 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:27.937842    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log" limit=1024
Sep 17 23:46:30.755282 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:30.755209    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6tvk2_a607de1c-e68e-4985-a7d6-0c1a27f4de8b/global-pull-secret-syncer/0.log"
Sep 17 23:46:30.836534 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:30.836490    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hbpfg_c9a28b22-04f6-4e93-a852-2e3cf1ce146f/konnectivity-agent/0.log"
Sep 17 23:46:30.917499 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:30.917471    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-236.ec2.internal_83f24f6f6691acab82204c1da59193da/haproxy/0.log"
Sep 17 23:46:37.042126 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:37.042093    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c479e3-8a7c-4388-b45b-0483ff8781e6/alertmanager/0.log"
Sep 17 23:46:37.062288 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:37.062258    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c479e3-8a7c-4388-b45b-0483ff8781e6/config-reloader/0.log"
Sep 17 23:46:37.084554 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:37.084528    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c479e3-8a7c-4388-b45b-0483ff8781e6/kube-rbac-proxy-web/0.log"
Sep 17 23:46:37.120055 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:37.120031    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c479e3-8a7c-4388-b45b-0483ff8781e6/kube-rbac-proxy/0.log"
Sep 17 23:46:37.143683 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:37.143665    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c479e3-8a7c-4388-b45b-0483ff8781e6/kube-rbac-proxy-metric/0.log"
Sep 17 23:46:37.172811 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:37.172795    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c479e3-8a7c-4388-b45b-0483ff8781e6/prom-label-proxy/0.log"
Sep 17 23:46:37.190089 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:37.190072    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c479e3-8a7c-4388-b45b-0483ff8781e6/init-config-reloader/0.log"
Sep 17 23:46:37.397494 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:37.397438    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-8b4g8_06b34b93-c3db-4c4e-a2f3-894e3b4093b6/monitoring-plugin/0.log"
Sep 17 23:46:37.523875 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:37.523847    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-spths_709b20a2-d606-4a1b-89ca-6225b474c6aa/node-exporter/0.log"
Sep 17 23:46:37.563394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:37.563377    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-spths_709b20a2-d606-4a1b-89ca-6225b474c6aa/kube-rbac-proxy/0.log"
Sep 17 23:46:37.589250 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:37.589233    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-spths_709b20a2-d606-4a1b-89ca-6225b474c6aa/init-textfile/0.log"
Sep 17 23:46:40.609917 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.609888    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"]
Sep 17 23:46:40.610282 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.610187    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7d6c6f6-3464-4212-8a1d-08cdb3d889ef" containerName="maas-api"
Sep 17 23:46:40.610282 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.610197    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7d6c6f6-3464-4212-8a1d-08cdb3d889ef" containerName="maas-api"
Sep 17 23:46:40.610282 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.610256    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7d6c6f6-3464-4212-8a1d-08cdb3d889ef" containerName="maas-api"
Sep 17 23:46:40.614384 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.614366    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.616418 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.616393    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kg5sm\"/\"openshift-service-ca.crt\""
Sep 17 23:46:40.617216 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.617197    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kg5sm\"/\"default-dockercfg-hwp27\""
Sep 17 23:46:40.617342 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.617217    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kg5sm\"/\"kube-root-ca.crt\""
Sep 17 23:46:40.621149 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.621125    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"]
Sep 17 23:46:40.763780 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.763757    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmvmw\" (UniqueName: \"kubernetes.io/projected/da50f4f6-e168-4121-8552-484e814b89b1-kube-api-access-bmvmw\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.763897 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.763784    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/da50f4f6-e168-4121-8552-484e814b89b1-proc\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.763897 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.763819    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da50f4f6-e168-4121-8552-484e814b89b1-lib-modules\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.763977 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.763888    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/da50f4f6-e168-4121-8552-484e814b89b1-podres\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.763977 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.763926    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da50f4f6-e168-4121-8552-484e814b89b1-sys\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.865196 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.865126    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmvmw\" (UniqueName: \"kubernetes.io/projected/da50f4f6-e168-4121-8552-484e814b89b1-kube-api-access-bmvmw\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.865196 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.865178    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/da50f4f6-e168-4121-8552-484e814b89b1-proc\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.865394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.865234    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da50f4f6-e168-4121-8552-484e814b89b1-lib-modules\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.865394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.865280    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/da50f4f6-e168-4121-8552-484e814b89b1-podres\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.865394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.865294    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/da50f4f6-e168-4121-8552-484e814b89b1-proc\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.865394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.865312    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da50f4f6-e168-4121-8552-484e814b89b1-sys\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.865394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.865336    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da50f4f6-e168-4121-8552-484e814b89b1-lib-modules\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.865394 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.865378    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/da50f4f6-e168-4121-8552-484e814b89b1-podres\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.865644 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.865403    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da50f4f6-e168-4121-8552-484e814b89b1-sys\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.872259 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.872240    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmvmw\" (UniqueName: \"kubernetes.io/projected/da50f4f6-e168-4121-8552-484e814b89b1-kube-api-access-bmvmw\") pod \"perf-node-gather-daemonset-m5rk5\" (UID: \"da50f4f6-e168-4121-8552-484e814b89b1\") " pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:40.924754 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:40.924728    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:41.194393 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:41.194364    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"]
Sep 17 23:46:41.195744 ip-10-0-137-236 kubenswrapper[2558]: W0917 23:46:41.195715    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podda50f4f6_e168_4121_8552_484e814b89b1.slice/crio-85bf3c0faf47e0d2e7a77f878768c5f5121626013a3df773f4e864b6e903feb7 WatchSource:0}: Error finding container 85bf3c0faf47e0d2e7a77f878768c5f5121626013a3df773f4e864b6e903feb7: Status 404 returned error can't find the container with id 85bf3c0faf47e0d2e7a77f878768c5f5121626013a3df773f4e864b6e903feb7
Sep 17 23:46:41.608244 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:41.608164    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5" event={"ID":"da50f4f6-e168-4121-8552-484e814b89b1","Type":"ContainerStarted","Data":"80445fd01de5665b203dca2fd3e9c886aefef9a8c48e882fa6c058b8d39111ad"}
Sep 17 23:46:41.608244 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:41.608212    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5" event={"ID":"da50f4f6-e168-4121-8552-484e814b89b1","Type":"ContainerStarted","Data":"85bf3c0faf47e0d2e7a77f878768c5f5121626013a3df773f4e864b6e903feb7"}
Sep 17 23:46:41.608244 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:41.608231    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:41.622833 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:41.622793    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5" podStartSLOduration=1.622780826 podStartE2EDuration="1.622780826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:46:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:46:41.622057771 +0000 UTC m=+2425.802696209" watchObservedRunningTime="2026-09-17 23:46:41.622780826 +0000 UTC m=+2425.803419263"
Sep 17 23:46:42.578170 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:42.578118    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t97lt_d6c1ec66-3187-4649-9205-78492b636e30/dns/0.log"
Sep 17 23:46:42.596921 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:42.596872    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t97lt_d6c1ec66-3187-4649-9205-78492b636e30/kube-rbac-proxy/0.log"
Sep 17 23:46:42.659842 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:42.659820    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nmb7f_0e34c616-c82c-4618-a991-5fb36612f2df/dns-node-resolver/0.log"
Sep 17 23:46:43.213142 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:43.213111    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cxgg6_6782274a-ac3d-41e1-ad5d-d221b7abb981/node-ca/0.log"
Sep 17 23:46:44.195964 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:44.195936    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx_17d06310-4705-4d83-a87d-c63d35283c59/istio-proxy/0.log"
Sep 17 23:46:44.519759 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:44.519687    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cz26c_f7ff21c7-43e7-4085-855d-03e61bd04eb5/discovery/0.log"
Sep 17 23:46:45.558100 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:45.558072    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wc6w2_73900d7a-0740-48c9-9a8d-5e576c1316b3/serve-healthcheck-canary/0.log"
Sep 17 23:46:46.124936 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:46.124912    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p2x2j_12d87332-b252-4f33-9f5f-27c8cc74f434/kube-rbac-proxy/0.log"
Sep 17 23:46:46.145847 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:46.145826    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p2x2j_12d87332-b252-4f33-9f5f-27c8cc74f434/exporter/0.log"
Sep 17 23:46:46.168243 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:46.168222    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p2x2j_12d87332-b252-4f33-9f5f-27c8cc74f434/extractor/0.log"
Sep 17 23:46:47.623500 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:47.623467    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kg5sm/perf-node-gather-daemonset-m5rk5"
Sep 17 23:46:48.639374 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:48.639343    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-r9ddr_85bee3c1-36a2-486a-bb72-b48cef7565c9/manager/0.log"
Sep 17 23:46:48.716954 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:48.716934    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-747fdf4bf5-t85kq_253192e6-6970-47fe-bba9-ab238e390a33/manager/0.log"
Sep 17 23:46:48.737510 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:48.737489    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-747fdf4bf5-t85kq_253192e6-6970-47fe-bba9-ab238e390a33/copy-manifests/0.log"
Sep 17 23:46:50.256774 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:50.256751    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-678fd6f85-z6b5t_5e158797-1120-4201-9b85-4e14d94295fc/manager/0.log"
Sep 17 23:46:50.844484 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:50.844413    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx_17d06310-4705-4d83-a87d-c63d35283c59/istio-proxy/0.log"
Sep 17 23:46:51.162357 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:51.162276    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cz26c_f7ff21c7-43e7-4085-855d-03e61bd04eb5/discovery/0.log"
Sep 17 23:46:52.754456 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:52.754431    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-v69lx_17d06310-4705-4d83-a87d-c63d35283c59/istio-proxy/0.log"
Sep 17 23:46:53.088784 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:53.088712    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cz26c_f7ff21c7-43e7-4085-855d-03e61bd04eb5/discovery/0.log"
Sep 17 23:46:57.046173 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:57.046144    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrtzv_6fdc916f-4430-45d4-8b5b-8bd8bfa42c75/kube-multus-additional-cni-plugins/0.log"
Sep 17 23:46:57.067703 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:57.067678    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrtzv_6fdc916f-4430-45d4-8b5b-8bd8bfa42c75/egress-router-binary-copy/0.log"
Sep 17 23:46:57.087365 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:57.087340    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrtzv_6fdc916f-4430-45d4-8b5b-8bd8bfa42c75/cni-plugins/0.log"
Sep 17 23:46:57.106591 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:57.106568    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrtzv_6fdc916f-4430-45d4-8b5b-8bd8bfa42c75/bond-cni-plugin/0.log"
Sep 17 23:46:57.132389 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:57.132368    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrtzv_6fdc916f-4430-45d4-8b5b-8bd8bfa42c75/routeoverride-cni/0.log"
Sep 17 23:46:57.153765 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:57.153745    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrtzv_6fdc916f-4430-45d4-8b5b-8bd8bfa42c75/whereabouts-cni-bincopy/0.log"
Sep 17 23:46:57.174505 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:57.174483    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zrtzv_6fdc916f-4430-45d4-8b5b-8bd8bfa42c75/whereabouts-cni/0.log"
Sep 17 23:46:57.210350 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:57.210330    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rvbth_9dbcbe63-94ef-4161-ba58-7a4a4349bd30/kube-multus/0.log"
Sep 17 23:46:57.323716 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:57.323666    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f4lpx_569bb01a-b894-4327-843d-9ed9959567c0/network-metrics-daemon/0.log"
Sep 17 23:46:57.342115 ip-10-0-137-236 kubenswrapper[2558]: I0917 23:46:57.342097    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f4lpx_569bb01a-b894-4327-843d-9ed9959567c0/kube-rbac-proxy/0.log"