Sep 11 13:04:34.933764 ip-10-0-143-79 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 13:04:35.327016 ip-10-0-143-79 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:04:35.327016 ip-10-0-143-79 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 13:04:35.327016 ip-10-0-143-79 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:04:35.327016 ip-10-0-143-79 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 13:04:35.327016 ip-10-0-143-79 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:04:35.329156 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.329062    2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 13:04:35.332809 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332788    2554 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:35.332809 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332809    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332814    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332818    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332820    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332823    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332826    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332835    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332839    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332843    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332845    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332848    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332851    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332853    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332856    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332859    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332862    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332865    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332868    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332870    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332873    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:35.332887 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332875    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332878    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332881    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332884    2554 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332887    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332890    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332892    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332896    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332898    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332901    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332904    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332907    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332909    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332912    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332914    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332917    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332919    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332922    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332924    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332928    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:35.333367 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332931    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332933    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332936    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332938    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332941    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332943    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332946    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332949    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332951    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332954    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332956    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332959    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332961    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332965    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332969    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332972    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332975    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332978    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332981    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332983    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:35.333891 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332986    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332988    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332991    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332993    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.332997    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333002    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333005    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333007    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333010    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333013    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333015    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333018    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333021    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333024    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333027    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333029    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333031    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333034    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333037    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:35.334384 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333039    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333042    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333045    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333047    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333050    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333053    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333055    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333060    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333063    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333067    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333071    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333075    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333079    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333082    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333085    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333088    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333091    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333094    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333096    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:35.334858 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333839    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333846    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333850    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333853    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333856    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333859    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333862    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333866    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333869    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333872    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333875    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333878    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333881    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333884    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333887    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333890    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333892    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333895    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333898    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333901    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:35.335315 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333905    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333908    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333911    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333913    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333916    2554 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333919    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333921    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333924    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333928    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333932    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333934    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333937    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333939    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333942    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333944    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333947    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333949    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333952    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333954    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333956    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:35.335908 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333960    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333962    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333965    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333967    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333970    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333972    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333975    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333977    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333980    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333983    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333985    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333988    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333992    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333994    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.333997    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334000    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334002    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334005    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334007    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334010    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:35.336778 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334012    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334016    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334019    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334021    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334024    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334026    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334029    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334031    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334034    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334036    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334039    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334042    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334044    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334047    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334050    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334052    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334055    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334057    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334059    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:35.337312 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334062    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334067    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334071    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334074    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334077    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334081    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334084    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334087    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334089    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334092    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334094    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334097    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334099    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334103    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334107    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334109    2554 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334112    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334115    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334118    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:35.337881 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334120    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334552    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334558    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334562    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334565    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334568    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334571    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334575    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334578    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334581    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334585    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334588    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334590    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334607    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334611    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334614    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334617    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334620    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334623    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:35.338343 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334626    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334629    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334631    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334634    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334637    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334639    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334642    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334645    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334649    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334653    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334656    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334659    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334661    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334664    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334666    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334669    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334672    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334675    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334677    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:35.338854 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334680    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334682    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334685    2554 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334688    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334691    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334693    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334696    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334698    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334701    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334704    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334707    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334709    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334715    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334718    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334720    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334723    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334726    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334728    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334731    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334733    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:35.339325 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334736    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334738    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334741    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334743    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334746    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334748    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334751    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334753    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334756    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334759    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334762    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334764    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334766    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334769    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334771    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334774    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334777    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334781    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334784    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334787    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:35.339837 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334790    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334793    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334796    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334799    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334803    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334806    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334809    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334812    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334815    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334818    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334820    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334823    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334826    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334828    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334831    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334833    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334836    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334839    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334841    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334844    2554 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:35.340362 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334846    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.334849    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.334917    2554 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.334955    2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.334962    2554 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.334966    2554 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.334970    2554 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.334973    2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.334978    2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.334983    2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.334987    2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.334990    2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.334995    2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.334999    2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335002    2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335005    2554 flags.go:64] FLAG: --cgroup-root=""
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335009    2554 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335013    2554 flags.go:64] FLAG: --client-ca-file=""
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335016    2554 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335019    2554 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335024    2554 flags.go:64] FLAG: --cluster-domain=""
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335027    2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335030    2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335033    2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 13:04:35.340864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335037    2554 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335041    2554 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335044    2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335047    2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335051    2554 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335054    2554 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335057    2554 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335060    2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335063    2554 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335066    2554 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335070    2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335073    2554 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335076    2554 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335079    2554 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335082    2554 flags.go:64] FLAG: --enable-server="true"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335085    2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335089    2554 flags.go:64] FLAG: --event-burst="100"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335093    2554 flags.go:64] FLAG: --event-qps="50"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335096    2554 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335099    2554 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335105    2554 flags.go:64] FLAG: --eviction-hard=""
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335109    2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335112    2554 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335115    2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335119    2554 flags.go:64] FLAG: --eviction-soft=""
Sep 11 13:04:35.341456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335123    2554 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335126    2554 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335129    2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335132    2554 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335136    2554 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335139    2554 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335142    2554 flags.go:64] FLAG: --feature-gates=""
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335145    2554 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335148    2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335152    2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335155    2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335158    2554 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335161    2554 flags.go:64] FLAG: --help="false"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335164    2554 flags.go:64] FLAG: --hostname-override="ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335168    2554 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335171    2554 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335174    2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335177    2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335181    2554 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335184    2554 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335186    2554 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335189    2554 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335192    2554 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 13:04:35.342072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335196    2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335199    2554 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335203    2554 flags.go:64] FLAG: --kube-reserved=""
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335206    2554 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335210    2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335213    2554 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335216    2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335219    2554 flags.go:64] FLAG: --lock-file=""
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335223    2554 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335226    2554 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335229    2554 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335235    2554 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335238    2554 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335241    2554 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335244    2554 flags.go:64] FLAG: --logging-format="text"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335247    2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335251    2554 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335254    2554 flags.go:64] FLAG: --manifest-url=""
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335257    2554 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335261    2554 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335264    2554 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335269    2554 flags.go:64] FLAG: --max-pods="110"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335272    2554 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335275    2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335278    2554 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 13:04:35.342679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335282    2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335285    2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335288    2554 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335291    2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335298    2554 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335301    2554 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335304    2554 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335307    2554 flags.go:64] FLAG: --pod-cidr=""
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335310    2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335315    2554 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335319    2554 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335322    2554 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335327    2554 flags.go:64] FLAG: --port="10250"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335330    2554 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335333    2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-016e3273b5f6e94dc"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335337    2554 flags.go:64] FLAG: --qos-reserved=""
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335340    2554 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335343    2554 flags.go:64] FLAG: --register-node="true"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335346    2554 flags.go:64] FLAG: --register-with-taints=""
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335350    2554 flags.go:64] FLAG: --registry-burst="10"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335353    2554 flags.go:64] FLAG: --registry-qps="5"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335356    2554 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335365    2554 flags.go:64] FLAG: --reserved-memory=""
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335369    2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335372    2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 13:04:35.343582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335375    2554 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335378    2554 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335381    2554 flags.go:64] FLAG: --runonce="false"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335384    2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335387    2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335390    2554 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335393    2554 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335396    2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335399    2554 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335403    2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335406    2554 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335409    2554 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335412    2554 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335415    2554 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335418    2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335421    2554 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335424    2554 flags.go:64] FLAG: --system-cgroups=""
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335427    2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335445    2554 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335450    2554 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335453    2554 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335458    2554 flags.go:64] FLAG: --tls-min-version=""
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335461    2554 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335464    2554 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335467    2554 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 13:04:35.344718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335470    2554 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335473    2554 flags.go:64] FLAG: --v="2"
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335478    2554 flags.go:64] FLAG: --version="false"
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335482    2554 flags.go:64] FLAG: --vmodule=""
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335486    2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.335489    2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336273    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336300    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336306    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336311    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336317    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336321    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336326    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336331    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336335    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336339    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336344    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336348    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336353    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336364    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336369    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:35.345643 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336373    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336378    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336382    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336387    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336392    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336396    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336401    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336405    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336410    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336415    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336420    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336429    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336433    2554 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336455    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336459    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336463    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336468    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336473    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336484    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336489    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:35.346304 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336493    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336497    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336502    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336511    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336515    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336520    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336524    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336528    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336532    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336540    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336547    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336554    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336560    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336565    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336569    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336578    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336583    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336587    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336590    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:35.347259 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336594    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336600    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336604    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336608    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336618    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336622    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336626    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336631    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336641    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336645    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336650    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336654    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336659    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336664    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336668    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336672    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336676    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336680    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336685    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336690    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:35.347953 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336694    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336705    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336738    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336744    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336750    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336755    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336760    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336765    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336771    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336775    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336780    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336784    2554 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336789    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336825    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336832    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336842    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336868    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336872    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336975    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336979    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:35.348733 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336984    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336987    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336990    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336993    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.336996    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.337570    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.345330    2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.345357    2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.346060    2554 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.346077    2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346177    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346187    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346193    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346198    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346202    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:35.349289 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346206    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346211    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346215    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346219    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346224    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346228    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346232    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346236    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346241    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346246    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346251    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346255    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346259    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346263    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346268    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346273    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346277    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346282    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346286    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:35.349773 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346290    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346296    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346301    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346307    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346313    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346318    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346323    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346327    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346332    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346337    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346343    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346349    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346355    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346361    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346365    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346370    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346374    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346378    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:35.350521 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346382    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346387    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346392    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346396    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346400    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346405    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346410    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346414    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346419    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346423    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346427    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346432    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346452    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346456    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346460    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346464    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346467    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346471    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346475    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346479    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:35.350975 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346483    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346487    2554 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346491    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346495    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346499    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346504    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346509    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346513    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346518    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346522    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346526    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346530    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346534    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346539    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346544    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346548    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346553    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346558    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346562    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346566    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:35.351853 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346571    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346575    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346581    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346586    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346591    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346595    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346599    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346604    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346608    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346613    2554 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346617    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346621    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346625    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346629    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346636    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346640    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:35.352576 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346644    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.346653    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346842    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346851    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346859    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346864    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346869    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346874    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346880    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346884    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346889    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346894    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346899    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346904    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346908    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:35.352978 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346913    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346917    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346921    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346925    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346930    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346935    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346939    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346943    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346947    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346952    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346956    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346960    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346964    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346968    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346972    2554 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346977    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346982    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346987    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346991    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346995    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.346999    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:35.353356 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347003    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347007    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347012    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347015    2554 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347020    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347024    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347028    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347033    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347037    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347041    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347045    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347049    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347054    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347058    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347063    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347067    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347071    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347076    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347080    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347084    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:35.353962 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347088    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347092    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347097    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347102    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347106    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347110    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347114    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347118    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347123    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347127    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347132    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347136    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347141    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347146    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347150    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347154    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347158    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347162    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347166    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347171    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:35.354557 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347203    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347210    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347216    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347220    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347226    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347230    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347235    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347239    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347244    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347248    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347252    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347256    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347260    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347264    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347268    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347272    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347276    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347280    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347287    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:35.355099 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347293    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:35.355654 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347299    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:35.355654 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347307    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:35.355654 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347312    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:35.355654 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347317    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:35.355654 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:35.347322    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:35.355654 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.347330    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:04:35.355654 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.347567    2554 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 13:04:35.355654 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.352096    2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 13:04:35.355654 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.353454    2554 server.go:1019] "Starting client certificate rotation"
Sep 11 13:04:35.355654 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.353557    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:04:35.355654 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.353602    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:04:35.356211 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.356192    2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:04:35.357496 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.357468    2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:04:35.368144 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.365861    2554 log.go:25] "Validated CRI v1 runtime API"
Sep 11 13:04:35.369994 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.369978    2554 log.go:25] "Validated CRI v1 image API"
Sep 11 13:04:35.370919 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.370901    2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 13:04:35.376607 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.376580    2554 fs.go:135] Filesystem UUIDs: map[1c946d56-bd55-4913-86f9-fd712b49e8e6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 900d3b01-9b2a-454b-bb92-7cfbfbc0ebe8:/dev/nvme0n1p4]
Sep 11 13:04:35.376676 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.376606    2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 13:04:35.383063 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.382916    2554 manager.go:216] Machine: {Timestamp:2026-09-11 13:04:35.380772245 +0000 UTC m=+0.331155637 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099760 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c2ab196ded6f7091c66c7c22cfe67 SystemUUID:ec2c2ab1-96de-d6f7-091c-66c7c22cfe67 BootID:c268f824-9040-4c19-86f6-b31b681aa568 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:af:a3:be:ca:71 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:af:a3:be:ca:71 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:b9:60:77:5d:d6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 13:04:35.383063 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.383050    2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 13:04:35.383213 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.383200    2554 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 13:04:35.386282 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.386248    2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 13:04:35.386486 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.386289    2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-79.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 13:04:35.387000 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.386986    2554 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 13:04:35.387000 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.387000    2554 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 13:04:35.387078 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.387009    2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:04:35.388209 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.388193    2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:04:35.388965 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.388955    2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 13:04:35.392408 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.392395    2554 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:04:35.392767 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.392755    2554 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 13:04:35.392813 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.392790    2554 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 13:04:35.392813 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.392811    2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 13:04:35.392892 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.392828    2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 13:04:35.392892 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.392838    2554 kubelet.go:398] "Adding apiserver pod source"
Sep 11 13:04:35.392892 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.392849    2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 13:04:35.393263 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.393248    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:04:35.393333 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.393276    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:04:35.393491 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.393471    2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:04:35.395429 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.395404    2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 13:04:35.397606 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.397591    2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 13:04:35.397673 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.397621    2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 13:04:35.399625 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.399610    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 13:04:35.399625 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.399629    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 13:04:35.399775 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.399635    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 13:04:35.399775 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.399641    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 13:04:35.399775 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.399650    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 13:04:35.399775 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.399656    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 13:04:35.399775 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.399669    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 13:04:35.399775 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.399675    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 13:04:35.399775 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.399682    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 13:04:35.399775 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.399688    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 13:04:35.399775 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.399697    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 13:04:35.399775 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.399702    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 13:04:35.400452 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.400429    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 13:04:35.400452 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.400452    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 13:04:35.404295 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.404281    2554 server.go:1268] "Started kubelet"
Sep 11 13:04:35.404390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.404364    2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 13:04:35.404427 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.404368    2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 13:04:35.404482 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.404468    2554 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 13:04:35.405152 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.405088    2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-79.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 13:04:35.405152 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.405131    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 13:04:35.405311 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.405165    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-79.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 13:04:35.405281 ip-10-0-143-79 systemd[1]: Started Kubernetes Kubelet.
Sep 11 13:04:35.405608 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.405593    2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 13:04:35.408157 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.406963    2554 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 13:04:35.409319 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.409295    2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 13:04:35.412494 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.412471    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 13:04:35.413143 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.413102    2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 13:04:35.413527 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.413490    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Sep 11 13:04:35.414011 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.413992    2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 13:04:35.414011 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.413997    2554 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 13:04:35.414141 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.414021    2554 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 13:04:35.414220 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.414184    2554 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 13:04:35.414220 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.414193    2554 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 13:04:35.414220 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.414208    2554 factory.go:55] Registering systemd factory
Sep 11 13:04:35.414339 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.414225    2554 factory.go:223] Registration of the systemd container factory successfully
Sep 11 13:04:35.414505 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.414486    2554 factory.go:153] Registering CRI-O factory
Sep 11 13:04:35.414582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.414509    2554 factory.go:223] Registration of the crio container factory successfully
Sep 11 13:04:35.414582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.414564    2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 13:04:35.414659 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.414583    2554 factory.go:103] Registering Raw factory
Sep 11 13:04:35.414659 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.414599    2554 manager.go:1196] Started watching for new ooms in manager
Sep 11 13:04:35.415006 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.414991    2554 manager.go:349] Starting recovery of all containers
Sep 11 13:04:35.415080 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.414071    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18d445f621cc9019  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-09-11 13:04:35.404255257 +0000 UTC m=+0.354638647,LastTimestamp:2026-09-11 13:04:35.404255257 +0000 UTC m=+0.354638647,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Sep 11 13:04:35.421855 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.421811    2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-79.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 13:04:35.422175 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.422147    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 13:04:35.424452 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.424388    2554 manager.go:354] Recovery completed
Sep 11 13:04:35.429472 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.429454    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:35.432140 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.432026    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:35.432140 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.432061    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:35.432140 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.432075    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:35.433402 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.432950    2554 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 13:04:35.433402 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.432962    2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 13:04:35.433402 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.432966    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:35.433402 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.432980    2554 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:04:35.433835 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.433814    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:35.433909 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.433853    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:35.433909 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.433872    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:35.434573 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.434426    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18d445f623748896  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-79.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-09-11 13:04:35.432040598 +0000 UTC m=+0.382423988,LastTimestamp:2026-09-11 13:04:35.432040598 +0000 UTC m=+0.382423988,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Sep 11 13:04:35.435085 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.435069    2554 policy_none.go:49] "None policy: Start"
Sep 11 13:04:35.435158 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.435091    2554 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 13:04:35.435158 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.435104    2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 13:04:35.436989 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.436972    2554 policy_none.go:47] "Start"
Sep 11 13:04:35.443553 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.443467    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18d445f62374f49a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-79.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-09-11 13:04:35.43206825 +0000 UTC m=+0.382451645,LastTimestamp:2026-09-11 13:04:35.43206825 +0000 UTC m=+0.382451645,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Sep 11 13:04:35.446863 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.446781    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18d445f623752167  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-79.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-09-11 13:04:35.432079719 +0000 UTC m=+0.382463109,LastTimestamp:2026-09-11 13:04:35.432079719 +0000 UTC m=+0.382463109,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Sep 11 13:04:35.453122 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.453031    2554 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-79.ec2.internal.18d445f623748896\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18d445f623748896  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-79.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-09-11 13:04:35.432040598 +0000 UTC m=+0.382423988,LastTimestamp:2026-09-11 13:04:35.433833864 +0000 UTC m=+0.384217259,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Sep 11 13:04:35.458896 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.458816    2554 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-79.ec2.internal.18d445f62374f49a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18d445f62374f49a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-79.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-09-11 13:04:35.43206825 +0000 UTC m=+0.382451645,LastTimestamp:2026-09-11 13:04:35.433861316 +0000 UTC m=+0.384244710,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Sep 11 13:04:35.463691 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.463599    2554 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-79.ec2.internal.18d445f623752167\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18d445f623752167  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-79.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-09-11 13:04:35.432079719 +0000 UTC m=+0.382463109,LastTimestamp:2026-09-11 13:04:35.433877182 +0000 UTC m=+0.384260572,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Sep 11 13:04:35.487991 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.464195    2554 manager.go:337] "Starting Device Plugin manager"
Sep 11 13:04:35.487991 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.464236    2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 13:04:35.487991 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.464246    2554 server.go:87] "Starting device plugin registration server"
Sep 11 13:04:35.487991 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.464723    2554 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 13:04:35.487991 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.464737    2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 13:04:35.487991 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.464829    2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 13:04:35.487991 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.464913    2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 13:04:35.487991 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.464921    2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 13:04:35.487991 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.465426    2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 13:04:35.487991 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.465506    2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-79.ec2.internal\" not found"
Sep 11 13:04:35.487991 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.483263    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18d445f626601f57  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-09-11 13:04:35.481034583 +0000 UTC m=+0.431417961,LastTimestamp:2026-09-11 13:04:35.481034583 +0000 UTC m=+0.431417961,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Sep 11 13:04:35.512498 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.512451    2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 13:04:35.513911 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.513891    2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 13:04:35.514007 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.513967    2554 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 13:04:35.514007 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.513998    2554 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 13:04:35.514181 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.514047    2554 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 13:04:35.526203 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.526164    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 11 13:04:35.565286 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.565253    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:35.566388 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.566367    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:35.566537 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.566402    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:35.566537 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.566419    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:35.566537 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.566465    2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.580008 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.579844    2554 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-79.ec2.internal.18d445f623748896\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18d445f623748896  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-79.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-09-11 13:04:35.432040598 +0000 UTC m=+0.382423988,LastTimestamp:2026-09-11 13:04:35.56638535 +0000 UTC m=+0.516768743,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Sep 11 13:04:35.581607 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.581587    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vvkw7"
Sep 11 13:04:35.592072 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.592042    2554 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-143-79.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.592356 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.592281    2554 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-79.ec2.internal.18d445f62374f49a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18d445f62374f49a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-79.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-09-11 13:04:35.43206825 +0000 UTC m=+0.382451645,LastTimestamp:2026-09-11 13:04:35.566409064 +0000 UTC m=+0.516792461,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Sep 11 13:04:35.594460 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.594418    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vvkw7"
Sep 11 13:04:35.598597 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.598513    2554 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-79.ec2.internal.18d445f623752167\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18d445f623752167  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-79.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-09-11 13:04:35.432079719 +0000 UTC m=+0.382463109,LastTimestamp:2026-09-11 13:04:35.566425068 +0000 UTC m=+0.516808460,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Sep 11 13:04:35.614461 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.614416    2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"]
Sep 11 13:04:35.614607 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.614524    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:35.615602 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.615588    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:35.615664 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.615618    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:35.615664 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.615629    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:35.618003 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.617989    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:35.618125 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.618110    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.618165 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.618147    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:35.618914 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.618899    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:35.618996 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.618924    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:35.618996 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.618933    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:35.618996 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.618899    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:35.619078 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.619007    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:35.619078 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.619019    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:35.621248 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.621233    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.621308 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.621263    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:35.622223 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.622205    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:35.622297 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.622235    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:35.622297 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.622248    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:35.639834 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.639795    2554 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-79.ec2.internal\" not found" node="ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.656469 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.656428    2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-79.ec2.internal\" not found" node="ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.661141 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.661124    2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-79.ec2.internal\" not found" node="ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.792741 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.792707    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:35.795553 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.795535    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:35.795641 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.795569    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:35.795641 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.795579    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:35.795641 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.795606    2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.804864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.804834    2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.804952 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.804867    2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-79.ec2.internal\": node \"ip-10-0-143-79.ec2.internal\" not found"
Sep 11 13:04:35.814840 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.814811    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7bf4c3f23d059c008c3e22fafba0aab7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal\" (UID: \"7bf4c3f23d059c008c3e22fafba0aab7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.814940 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.814844    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bf4c3f23d059c008c3e22fafba0aab7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal\" (UID: \"7bf4c3f23d059c008c3e22fafba0aab7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.814940 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.814868    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9c13e4afb5118b6c262250604375f150-config\") pod \"kube-apiserver-proxy-ip-10-0-143-79.ec2.internal\" (UID: \"9c13e4afb5118b6c262250604375f150\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.850831 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.850763    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Sep 11 13:04:35.915301 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.915266    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9c13e4afb5118b6c262250604375f150-config\") pod \"kube-apiserver-proxy-ip-10-0-143-79.ec2.internal\" (UID: \"9c13e4afb5118b6c262250604375f150\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.915301 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.915301    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7bf4c3f23d059c008c3e22fafba0aab7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal\" (UID: \"7bf4c3f23d059c008c3e22fafba0aab7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.915424 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.915324    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bf4c3f23d059c008c3e22fafba0aab7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal\" (UID: \"7bf4c3f23d059c008c3e22fafba0aab7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.915424 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.915382    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9c13e4afb5118b6c262250604375f150-config\") pod \"kube-apiserver-proxy-ip-10-0-143-79.ec2.internal\" (UID: \"9c13e4afb5118b6c262250604375f150\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.915424 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.915413    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bf4c3f23d059c008c3e22fafba0aab7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal\" (UID: \"7bf4c3f23d059c008c3e22fafba0aab7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.915550 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.915385    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7bf4c3f23d059c008c3e22fafba0aab7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal\" (UID: \"7bf4c3f23d059c008c3e22fafba0aab7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.951433 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:35.951401    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Sep 11 13:04:35.959581 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.959554    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:35.963303 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:35.963283    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:37.685119 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:37.685091    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Sep 11 13:04:37.785641 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:37.785557    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Sep 11 13:04:37.886134 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:37.886101    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Sep 11 13:04:37.924513 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:37.924483    2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:37.986517 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:37.986484    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Sep 11 13:04:37.986517 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:37.986495    2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 13:04:37.987035 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:37.986694    2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 13:04:38.030876 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.030839    2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:38.087408 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:38.087177    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Sep 11 13:04:38.153898 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.153871    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:38.154893 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.154870    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:38.155001 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.154907    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:38.155001 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.154921    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:38.159594 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:38.159556    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bf4c3f23d059c008c3e22fafba0aab7.slice/crio-0670f3a254a10d2d444d4baccb984242f454465df63d335cfd26836dc04680f3 WatchSource:0}: Error finding container 0670f3a254a10d2d444d4baccb984242f454465df63d335cfd26836dc04680f3: Status 404 returned error can't find the container with id 0670f3a254a10d2d444d4baccb984242f454465df63d335cfd26836dc04680f3
Sep 11 13:04:38.160137 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:38.160113    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c13e4afb5118b6c262250604375f150.slice/crio-2609b2065b2d30a44f8b5a406d91e807a605ba4025bbce8b4c4adbffa9576d6c WatchSource:0}: Error finding container 2609b2065b2d30a44f8b5a406d91e807a605ba4025bbce8b4c4adbffa9576d6c: Status 404 returned error can't find the container with id 2609b2065b2d30a44f8b5a406d91e807a605ba4025bbce8b4c4adbffa9576d6c
Sep 11 13:04:38.167382 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.167351    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:04:38.187447 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:38.187416    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Sep 11 13:04:38.232264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.232224    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 12:59:37 +0000 UTC" deadline="2028-03-20 20:45:27.268932868 +0000 UTC"
Sep 11 13:04:38.232264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.232258    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13351h40m49.036679s"
Sep 11 13:04:38.268633 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.268606    2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:38.346816 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.346743    2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:38.365249 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.365225    2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:04:38.366153 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.366138    2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"
Sep 11 13:04:38.378528 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.378506    2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:04:38.727722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:38.727474    2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:39.027192 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.027116    2554 apiserver.go:52] "Watching apiserver"
Sep 11 13:04:39.036176 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.036141    2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 13:04:39.037273 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.037239    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal","openshift-cluster-node-tuning-operator/tuned-nlfvh","openshift-dns/node-resolver-twqp9","openshift-image-registry/node-ca-x89jp","openshift-multus/multus-additional-cni-plugins-gt6zk","openshift-multus/network-metrics-daemon-9rsjp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal","openshift-multus/multus-l6vms","openshift-network-diagnostics/network-check-target-qbm58","openshift-network-operator/iptables-alerter-bq4dz","openshift-ovn-kubernetes/ovnkube-node-46vhh","kube-system/global-pull-secret-syncer-p5hww","kube-system/konnectivity-agent-8hpxh"]
Sep 11 13:04:39.039929 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.039905    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.041995 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.041977    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:39.042097 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.042043    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:04:39.042872 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.042856    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 13:04:39.043145 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.043120    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2vr42\""
Sep 11 13:04:39.043265 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.043176    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 13:04:39.043398 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.043383    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 13:04:39.043979 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.043966    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-twqp9"
Sep 11 13:04:39.045957 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.045939    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 13:04:39.046852 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.046837    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x89jp"
Sep 11 13:04:39.049318 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.049291    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.051622 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.051604    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:39.051715 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.051677    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:04:39.054089 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.054069    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.055183 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.055163    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 13:04:39.056051 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056029    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfzvg\" (UniqueName: \"kubernetes.io/projected/1572be9a-6852-40f5-951c-b0a82cd9961f-kube-api-access-hfzvg\") pod \"node-ca-x89jp\" (UID: \"1572be9a-6852-40f5-951c-b0a82cd9961f\") " pod="openshift-image-registry/node-ca-x89jp"
Sep 11 13:04:39.056134 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056061    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.056134 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056095    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6942647-056e-4b32-a2b4-ea2e90f7be76-kubelet-config\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:39.056225 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056134    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6942647-056e-4b32-a2b4-ea2e90f7be76-dbus\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:39.056225 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056166    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvf79\" (UniqueName: \"kubernetes.io/projected/3b49a550-e226-4de1-8351-a9fd45b4f76f-kube-api-access-nvf79\") pod \"node-resolver-twqp9\" (UID: \"3b49a550-e226-4de1-8351-a9fd45b4f76f\") " pod="openshift-dns/node-resolver-twqp9"
Sep 11 13:04:39.056225 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056202    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1572be9a-6852-40f5-951c-b0a82cd9961f-serviceca\") pod \"node-ca-x89jp\" (UID: \"1572be9a-6852-40f5-951c-b0a82cd9961f\") " pod="openshift-image-registry/node-ca-x89jp"
Sep 11 13:04:39.056380 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056245    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b0312884-f43d-4476-8501-83abe961b238-cnibin\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.056380 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056276    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b0312884-f43d-4476-8501-83abe961b238-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.056380 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056305    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8hpxh"
Sep 11 13:04:39.056380 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056315    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-socket-dir\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.056380 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056337    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-device-dir\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.056380 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056354    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-sys-fs\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.056656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056390    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh2cb\" (UniqueName: \"kubernetes.io/projected/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-kube-api-access-mh2cb\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.056656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056407    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b49a550-e226-4de1-8351-a9fd45b4f76f-hosts-file\") pod \"node-resolver-twqp9\" (UID: \"3b49a550-e226-4de1-8351-a9fd45b4f76f\") " pod="openshift-dns/node-resolver-twqp9"
Sep 11 13:04:39.056656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056421    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b0312884-f43d-4476-8501-83abe961b238-cni-binary-copy\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.056656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056438    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b0312884-f43d-4476-8501-83abe961b238-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.056656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056464    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b0312884-f43d-4476-8501-83abe961b238-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.056656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056509    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-registration-dir\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.056656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056551    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1572be9a-6852-40f5-951c-b0a82cd9961f-host\") pod \"node-ca-x89jp\" (UID: \"1572be9a-6852-40f5-951c-b0a82cd9961f\") " pod="openshift-image-registry/node-ca-x89jp"
Sep 11 13:04:39.056656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056580    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0312884-f43d-4476-8501-83abe961b238-system-cni-dir\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.056656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056603    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4khn\" (UniqueName: \"kubernetes.io/projected/b0312884-f43d-4476-8501-83abe961b238-kube-api-access-s4khn\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.056656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056628    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.056656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056644    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:39.056656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.056660    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b49a550-e226-4de1-8351-a9fd45b4f76f-tmp-dir\") pod \"node-resolver-twqp9\" (UID: \"3b49a550-e226-4de1-8351-a9fd45b4f76f\") " pod="openshift-dns/node-resolver-twqp9"
Sep 11 13:04:39.057150 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.057132    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 13:04:39.057486 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.057466    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 13:04:39.057576 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.057541    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 13:04:39.057706 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.057691    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 13:04:39.057797 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.057761    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 13:04:39.057920 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.057904    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wv54k\""
Sep 11 13:04:39.057992 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.057962    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ntz2p\""
Sep 11 13:04:39.057992 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.057985    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 13:04:39.058099 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.058036    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 13:04:39.058148 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.058118    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 13:04:39.058197 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.058166    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 13:04:39.058301 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.058231    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wv67q\""
Sep 11 13:04:39.058301 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.058233    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 13:04:39.058415 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.058341    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g6mrr\""
Sep 11 13:04:39.058689 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.058666    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 13:04:39.058756 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.058737    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-shqhz\""
Sep 11 13:04:39.059000 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.058979    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.060487 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.060466    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 13:04:39.061401 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.061381    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:39.061498 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.061471    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:04:39.064097 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.064077    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bq4dz"
Sep 11 13:04:39.066070 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.066048    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:04:39.066167 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.066092    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 13:04:39.067014 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.066992    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:04:39.067136 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.067116    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.067264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.067246    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 13:04:39.067324 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.067293    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4x8ff\""
Sep 11 13:04:39.068100 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.067980    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sj5j4\""
Sep 11 13:04:39.068573 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.068555    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 13:04:39.072579 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.072539    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 13:04:39.072579 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.072543    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 13:04:39.072748 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.072692    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 13:04:39.072873 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.072851    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 13:04:39.073219 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.073199    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 13:04:39.073799 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.073257    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dvkdh\""
Sep 11 13:04:39.075794 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.075774    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 13:04:39.136286 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.136245    2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:04:39.136485 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.136327    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fh5mz"
Sep 11 13:04:39.147114 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.147079    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fh5mz"
Sep 11 13:04:39.148203 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.148172    2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 13:04:39.155041 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.154989    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal" event={"ID":"7bf4c3f23d059c008c3e22fafba0aab7","Type":"ContainerStarted","Data":"0670f3a254a10d2d444d4baccb984242f454465df63d335cfd26836dc04680f3"}
Sep 11 13:04:39.156205 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.156182    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal" event={"ID":"9c13e4afb5118b6c262250604375f150","Type":"ContainerStarted","Data":"2609b2065b2d30a44f8b5a406d91e807a605ba4025bbce8b4c4adbffa9576d6c"}
Sep 11 13:04:39.157421 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157391    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b49a550-e226-4de1-8351-a9fd45b4f76f-tmp-dir\") pod \"node-resolver-twqp9\" (UID: \"3b49a550-e226-4de1-8351-a9fd45b4f76f\") " pod="openshift-dns/node-resolver-twqp9"
Sep 11 13:04:39.157532 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157491    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-sys\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.157586 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157527    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b0312884-f43d-4476-8501-83abe961b238-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.157586 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157549    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-systemd\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.157680 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157599    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-socket-dir\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.157752 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157731    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-sys-fs\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.157800 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157746    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b49a550-e226-4de1-8351-a9fd45b4f76f-tmp-dir\") pod \"node-resolver-twqp9\" (UID: \"3b49a550-e226-4de1-8351-a9fd45b4f76f\") " pod="openshift-dns/node-resolver-twqp9"
Sep 11 13:04:39.157800 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157785    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-run-openvswitch\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.157891 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157812    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-run-ovn\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.157891 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157785    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-socket-dir\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.157891 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157858    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-sys-fs\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.157891 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157869    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-log-socket\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.158034 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157899    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-run\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.158034 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157924    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-system-cni-dir\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.158034 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157947    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-run-systemd\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.158034 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157968    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-node-log\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.158034 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.157996    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.158034 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158021    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfzvg\" (UniqueName: \"kubernetes.io/projected/1572be9a-6852-40f5-951c-b0a82cd9961f-kube-api-access-hfzvg\") pod \"node-ca-x89jp\" (UID: \"1572be9a-6852-40f5-951c-b0a82cd9961f\") " pod="openshift-image-registry/node-ca-x89jp"
Sep 11 13:04:39.158266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158045    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-run-k8s-cni-cncf-io\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.158266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158070    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-var-lib-cni-bin\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.158266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158098    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-var-lib-kubelet\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.158266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158124    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b0312884-f43d-4476-8501-83abe961b238-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.158266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158137    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b0312884-f43d-4476-8501-83abe961b238-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.158266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158152    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b0312884-f43d-4476-8501-83abe961b238-cnibin\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.158266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158155    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.158266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158202    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-var-lib-cni-multus\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.158266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158228    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b49a550-e226-4de1-8351-a9fd45b4f76f-hosts-file\") pod \"node-resolver-twqp9\" (UID: \"3b49a550-e226-4de1-8351-a9fd45b4f76f\") " pod="openshift-dns/node-resolver-twqp9"
Sep 11 13:04:39.158266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158225    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b0312884-f43d-4476-8501-83abe961b238-cnibin\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.158266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158254    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b0312884-f43d-4476-8501-83abe961b238-cni-binary-copy\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158280    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b0312884-f43d-4476-8501-83abe961b238-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158292    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b49a550-e226-4de1-8351-a9fd45b4f76f-hosts-file\") pod \"node-resolver-twqp9\" (UID: \"3b49a550-e226-4de1-8351-a9fd45b4f76f\") " pod="openshift-dns/node-resolver-twqp9"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158304    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88fb6546-390a-48f2-a61f-93bce22f0c64-ovn-node-metrics-cert\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158328    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-run-netns\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158354    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1572be9a-6852-40f5-951c-b0a82cd9961f-host\") pod \"node-ca-x89jp\" (UID: \"1572be9a-6852-40f5-951c-b0a82cd9961f\") " pod="openshift-image-registry/node-ca-x89jp"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158403    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4khn\" (UniqueName: \"kubernetes.io/projected/b0312884-f43d-4476-8501-83abe961b238-kube-api-access-s4khn\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158431    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1572be9a-6852-40f5-951c-b0a82cd9961f-host\") pod \"node-ca-x89jp\" (UID: \"1572be9a-6852-40f5-951c-b0a82cd9961f\") " pod="openshift-image-registry/node-ca-x89jp"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158477    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgwsk\" (UniqueName: \"kubernetes.io/projected/0498fed7-8022-4a6b-9e69-3a560c5f5e02-kube-api-access-dgwsk\") pod \"network-metrics-daemon-9rsjp\" (UID: \"0498fed7-8022-4a6b-9e69-3a560c5f5e02\") " pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158504    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-sysconfig\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158528    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b0312884-f43d-4476-8501-83abe961b238-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158625    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b0312884-f43d-4476-8501-83abe961b238-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158667    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-sysctl-d\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158703    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-var-lib-kubelet\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158727    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7d0e0990-ad45-4112-823a-dad4edc125bf-iptables-alerter-script\") pod \"iptables-alerter-bq4dz\" (UID: \"7d0e0990-ad45-4112-823a-dad4edc125bf\") " pod="openshift-network-operator/iptables-alerter-bq4dz"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158754    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-hostroot\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.158803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158800    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158839    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-cni-netd\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158855    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b0312884-f43d-4476-8501-83abe961b238-cni-binary-copy\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158865    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-tuned\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158888    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-multus-conf-dir\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.158934    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158952    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-etc-kubernetes\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158981    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/00d20a97-d950-479a-9e5d-0ebb126e3e9c-agent-certs\") pod \"konnectivity-agent-8hpxh\" (UID: \"00d20a97-d950-479a-9e5d-0ebb126e3e9c\") " pod="kube-system/konnectivity-agent-8hpxh"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.158997    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-systemd-units\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.159023    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret podName:c6942647-056e-4b32-a2b4-ea2e90f7be76 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:39.659005842 +0000 UTC m=+2.976462910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret") pod "global-pull-secret-syncer-p5hww" (UID: "c6942647-056e-4b32-a2b4-ea2e90f7be76") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159039    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6942647-056e-4b32-a2b4-ea2e90f7be76-kubelet-config\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159066    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-cni-bin\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159090    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-multus-cni-dir\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159117    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8320a6e-2b9f-4253-8549-bb485d077d99-multus-daemon-config\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159143    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6942647-056e-4b32-a2b4-ea2e90f7be76-kubelet-config\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159143    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ntnk\" (UniqueName: \"kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk\") pod \"network-check-target-qbm58\" (UID: \"439302b6-1aa6-4530-9e4f-3534c305b8e1\") " pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:39.159593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159189    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs\") pod \"network-metrics-daemon-9rsjp\" (UID: \"0498fed7-8022-4a6b-9e69-3a560c5f5e02\") " pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159214    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-host\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159234    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrfq9\" (UniqueName: \"kubernetes.io/projected/646e971b-8337-4e59-a5fa-f01eed3c8e33-kube-api-access-wrfq9\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159253    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8320a6e-2b9f-4253-8549-bb485d077d99-cni-binary-copy\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159274    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-run-multus-certs\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159299    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-registration-dir\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159322    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0312884-f43d-4476-8501-83abe961b238-system-cni-dir\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159337    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-kubelet\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159352    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-etc-openvswitch\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159386    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-kubernetes\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159389    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-registration-dir\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159411    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0312884-f43d-4476-8501-83abe961b238-system-cni-dir\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159428    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-run-netns\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159456    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-var-lib-openvswitch\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159504    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-run-ovn-kubernetes\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159550    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-sysctl-conf\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.160294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159579    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159599    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6942647-056e-4b32-a2b4-ea2e90f7be76-dbus\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159631    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvf79\" (UniqueName: \"kubernetes.io/projected/3b49a550-e226-4de1-8351-a9fd45b4f76f-kube-api-access-nvf79\") pod \"node-resolver-twqp9\" (UID: \"3b49a550-e226-4de1-8351-a9fd45b4f76f\") " pod="openshift-dns/node-resolver-twqp9"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159668    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1572be9a-6852-40f5-951c-b0a82cd9961f-serviceca\") pod \"node-ca-x89jp\" (UID: \"1572be9a-6852-40f5-951c-b0a82cd9961f\") " pod="openshift-image-registry/node-ca-x89jp"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159684    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88fb6546-390a-48f2-a61f-93bce22f0c64-ovnkube-config\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159704    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-lib-modules\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159721    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d0e0990-ad45-4112-823a-dad4edc125bf-host-slash\") pod \"iptables-alerter-bq4dz\" (UID: \"7d0e0990-ad45-4112-823a-dad4edc125bf\") " pod="openshift-network-operator/iptables-alerter-bq4dz"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159741    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-device-dir\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159766    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh2cb\" (UniqueName: \"kubernetes.io/projected/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-kube-api-access-mh2cb\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159789    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88fb6546-390a-48f2-a61f-93bce22f0c64-env-overrides\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159800    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6942647-056e-4b32-a2b4-ea2e90f7be76-dbus\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159811    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znzrm\" (UniqueName: \"kubernetes.io/projected/7d0e0990-ad45-4112-823a-dad4edc125bf-kube-api-access-znzrm\") pod \"iptables-alerter-bq4dz\" (UID: \"7d0e0990-ad45-4112-823a-dad4edc125bf\") " pod="openshift-network-operator/iptables-alerter-bq4dz"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159816    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-device-dir\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159703    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159835    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-cnibin\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159866    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-multus-socket-dir-parent\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159888    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twxkm\" (UniqueName: \"kubernetes.io/projected/c8320a6e-2b9f-4253-8549-bb485d077d99-kube-api-access-twxkm\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.161108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159902    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/00d20a97-d950-479a-9e5d-0ebb126e3e9c-konnectivity-ca\") pod \"konnectivity-agent-8hpxh\" (UID: \"00d20a97-d950-479a-9e5d-0ebb126e3e9c\") " pod="kube-system/konnectivity-agent-8hpxh"
Sep 11 13:04:39.161913 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159920    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-slash\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.161913 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159939    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.161913 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159962    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88fb6546-390a-48f2-a61f-93bce22f0c64-ovnkube-script-lib\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.161913 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.159980    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98ghk\" (UniqueName: \"kubernetes.io/projected/88fb6546-390a-48f2-a61f-93bce22f0c64-kube-api-access-98ghk\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.161913 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.160014    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-modprobe-d\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.161913 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.160032    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/646e971b-8337-4e59-a5fa-f01eed3c8e33-tmp\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.161913 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.160098    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1572be9a-6852-40f5-951c-b0a82cd9961f-serviceca\") pod \"node-ca-x89jp\" (UID: \"1572be9a-6852-40f5-951c-b0a82cd9961f\") " pod="openshift-image-registry/node-ca-x89jp"
Sep 11 13:04:39.177631 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.177598    2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 13:04:39.181963 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.181895    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfzvg\" (UniqueName: \"kubernetes.io/projected/1572be9a-6852-40f5-951c-b0a82cd9961f-kube-api-access-hfzvg\") pod \"node-ca-x89jp\" (UID: \"1572be9a-6852-40f5-951c-b0a82cd9961f\") " pod="openshift-image-registry/node-ca-x89jp"
Sep 11 13:04:39.181963 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.181920    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvf79\" (UniqueName: \"kubernetes.io/projected/3b49a550-e226-4de1-8351-a9fd45b4f76f-kube-api-access-nvf79\") pod \"node-resolver-twqp9\" (UID: \"3b49a550-e226-4de1-8351-a9fd45b4f76f\") " pod="openshift-dns/node-resolver-twqp9"
Sep 11 13:04:39.184267 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.184243    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4khn\" (UniqueName: \"kubernetes.io/projected/b0312884-f43d-4476-8501-83abe961b238-kube-api-access-s4khn\") pod \"multus-additional-cni-plugins-gt6zk\" (UID: \"b0312884-f43d-4476-8501-83abe961b238\") " pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.201892 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.201858    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh2cb\" (UniqueName: \"kubernetes.io/projected/5ded15f4-81ba-411c-8cc0-1b3a31612e4e-kube-api-access-mh2cb\") pod \"aws-ebs-csi-driver-node-xxhlt\" (UID: \"5ded15f4-81ba-411c-8cc0-1b3a31612e4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.260689 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.260650    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88fb6546-390a-48f2-a61f-93bce22f0c64-ovnkube-config\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.260689 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.260692    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-lib-modules\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.260961 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.260711    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d0e0990-ad45-4112-823a-dad4edc125bf-host-slash\") pod \"iptables-alerter-bq4dz\" (UID: \"7d0e0990-ad45-4112-823a-dad4edc125bf\") " pod="openshift-network-operator/iptables-alerter-bq4dz"
Sep 11 13:04:39.260961 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.260743    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88fb6546-390a-48f2-a61f-93bce22f0c64-env-overrides\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.260961 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.260847    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znzrm\" (UniqueName: \"kubernetes.io/projected/7d0e0990-ad45-4112-823a-dad4edc125bf-kube-api-access-znzrm\") pod \"iptables-alerter-bq4dz\" (UID: \"7d0e0990-ad45-4112-823a-dad4edc125bf\") " pod="openshift-network-operator/iptables-alerter-bq4dz"
Sep 11 13:04:39.260961 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.260865    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-cnibin\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.260961 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.260881    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-multus-socket-dir-parent\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.260961 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.260908    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twxkm\" (UniqueName: \"kubernetes.io/projected/c8320a6e-2b9f-4253-8549-bb485d077d99-kube-api-access-twxkm\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.260961 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.260922    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/00d20a97-d950-479a-9e5d-0ebb126e3e9c-konnectivity-ca\") pod \"konnectivity-agent-8hpxh\" (UID: \"00d20a97-d950-479a-9e5d-0ebb126e3e9c\") " pod="kube-system/konnectivity-agent-8hpxh"
Sep 11 13:04:39.260961 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.260921    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-lib-modules\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.260961 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.260951    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-cnibin\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.261390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.260968    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-slash\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.261390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.260886    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d0e0990-ad45-4112-823a-dad4edc125bf-host-slash\") pod \"iptables-alerter-bq4dz\" (UID: \"7d0e0990-ad45-4112-823a-dad4edc125bf\") " pod="openshift-network-operator/iptables-alerter-bq4dz"
Sep 11 13:04:39.261390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261006    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.261390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261034    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-multus-socket-dir-parent\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.261390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261083    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88fb6546-390a-48f2-a61f-93bce22f0c64-ovnkube-script-lib\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.261390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261138    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98ghk\" (UniqueName: \"kubernetes.io/projected/88fb6546-390a-48f2-a61f-93bce22f0c64-kube-api-access-98ghk\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.261390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261278    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-modprobe-d\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.261390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261318    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/646e971b-8337-4e59-a5fa-f01eed3c8e33-tmp\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.261390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261344    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-sys\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.261390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261385    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-systemd\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261412    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-run-openvswitch\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261436    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-run-ovn\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261454    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-modprobe-d\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261460    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-log-socket\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261529    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-run\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261559    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-system-cni-dir\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261584    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-run-systemd\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261608    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-node-log\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261634    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-run-k8s-cni-cncf-io\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261657    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-var-lib-cni-bin\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261682    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-var-lib-kubelet\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261703    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-systemd\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261700    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88fb6546-390a-48f2-a61f-93bce22f0c64-env-overrides\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261713    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-var-lib-cni-multus\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261743    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88fb6546-390a-48f2-a61f-93bce22f0c64-ovn-node-metrics-cert\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261745    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-run-openvswitch\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261502    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-log-socket\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.261859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261758    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-sys\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261656    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88fb6546-390a-48f2-a61f-93bce22f0c64-ovnkube-config\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261790    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-run-netns\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261792    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/00d20a97-d950-479a-9e5d-0ebb126e3e9c-konnectivity-ca\") pod \"konnectivity-agent-8hpxh\" (UID: \"00d20a97-d950-479a-9e5d-0ebb126e3e9c\") " pod="kube-system/konnectivity-agent-8hpxh"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261170    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261820    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-var-lib-kubelet\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261876    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-var-lib-cni-bin\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261867    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-node-log\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261918    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-run-netns\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261877    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgwsk\" (UniqueName: \"kubernetes.io/projected/0498fed7-8022-4a6b-9e69-3a560c5f5e02-kube-api-access-dgwsk\") pod \"network-metrics-daemon-9rsjp\" (UID: \"0498fed7-8022-4a6b-9e69-3a560c5f5e02\") " pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261917    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-var-lib-cni-multus\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.261949    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-slash\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262073    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-run-systemd\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262097    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-system-cni-dir\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262110    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-sysconfig\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262118    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-run\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262156    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-run-ovn\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262141    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-run-k8s-cni-cncf-io\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.262722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262205    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-sysconfig\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262240    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-sysctl-d\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262286    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-var-lib-kubelet\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262318    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-var-lib-kubelet\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262353    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7d0e0990-ad45-4112-823a-dad4edc125bf-iptables-alerter-script\") pod \"iptables-alerter-bq4dz\" (UID: \"7d0e0990-ad45-4112-823a-dad4edc125bf\") " pod="openshift-network-operator/iptables-alerter-bq4dz"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262397    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-hostroot\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262442    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-hostroot\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262508    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-sysctl-d\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262550    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-cni-netd\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262514    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-cni-netd\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262614    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-tuned\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262672    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88fb6546-390a-48f2-a61f-93bce22f0c64-ovnkube-script-lib\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262681    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-multus-conf-dir\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262723    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-multus-conf-dir\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262748    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-etc-kubernetes\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262815    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/00d20a97-d950-479a-9e5d-0ebb126e3e9c-agent-certs\") pod \"konnectivity-agent-8hpxh\" (UID: \"00d20a97-d950-479a-9e5d-0ebb126e3e9c\") " pod="kube-system/konnectivity-agent-8hpxh"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262844    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-systemd-units\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262902    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-cni-bin\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.263584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262926    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-multus-cni-dir\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.262950    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8320a6e-2b9f-4253-8549-bb485d077d99-multus-daemon-config\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263055    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ntnk\" (UniqueName: \"kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk\") pod \"network-check-target-qbm58\" (UID: \"439302b6-1aa6-4530-9e4f-3534c305b8e1\") " pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263066    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7d0e0990-ad45-4112-823a-dad4edc125bf-iptables-alerter-script\") pod \"iptables-alerter-bq4dz\" (UID: \"7d0e0990-ad45-4112-823a-dad4edc125bf\") " pod="openshift-network-operator/iptables-alerter-bq4dz"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263083    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs\") pod \"network-metrics-daemon-9rsjp\" (UID: \"0498fed7-8022-4a6b-9e69-3a560c5f5e02\") " pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263157    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-host\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.263167    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263189    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrfq9\" (UniqueName: \"kubernetes.io/projected/646e971b-8337-4e59-a5fa-f01eed3c8e33-kube-api-access-wrfq9\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263258    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-cni-bin\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.263256    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs podName:0498fed7-8022-4a6b-9e69-3a560c5f5e02 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:39.763206042 +0000 UTC m=+3.080663082 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs") pod "network-metrics-daemon-9rsjp" (UID: "0498fed7-8022-4a6b-9e69-3a560c5f5e02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263289    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8320a6e-2b9f-4253-8549-bb485d077d99-cni-binary-copy\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263318    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-run-multus-certs\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263464    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-multus-cni-dir\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263483    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-host\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263560    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-host-run-multus-certs\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263603    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-kubelet\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263633    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-etc-openvswitch\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.264439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263688    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-kubernetes\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263717    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-run-netns\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263742    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-var-lib-openvswitch\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263785    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-run-ovn-kubernetes\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263853    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-sysctl-conf\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.263874    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8320a6e-2b9f-4253-8549-bb485d077d99-cni-binary-copy\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.264023    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-sysctl-conf\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.264042    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8320a6e-2b9f-4253-8549-bb485d077d99-multus-daemon-config\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.264072    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-kubelet\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.264096    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-systemd-units\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.264098    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-run-ovn-kubernetes\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.264112    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-kubernetes\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.264129    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-etc-openvswitch\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.264139    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8320a6e-2b9f-4253-8549-bb485d077d99-etc-kubernetes\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.264145    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-host-run-netns\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.264264    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88fb6546-390a-48f2-a61f-93bce22f0c64-var-lib-openvswitch\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.264857    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/646e971b-8337-4e59-a5fa-f01eed3c8e33-tmp\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.265128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.264907    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88fb6546-390a-48f2-a61f-93bce22f0c64-ovn-node-metrics-cert\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.266426 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.266404    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/646e971b-8337-4e59-a5fa-f01eed3c8e33-etc-tuned\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.266640 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.266621    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/00d20a97-d950-479a-9e5d-0ebb126e3e9c-agent-certs\") pod \"konnectivity-agent-8hpxh\" (UID: \"00d20a97-d950-479a-9e5d-0ebb126e3e9c\") " pod="kube-system/konnectivity-agent-8hpxh"
Sep 11 13:04:39.278836 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.278763    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znzrm\" (UniqueName: \"kubernetes.io/projected/7d0e0990-ad45-4112-823a-dad4edc125bf-kube-api-access-znzrm\") pod \"iptables-alerter-bq4dz\" (UID: \"7d0e0990-ad45-4112-823a-dad4edc125bf\") " pod="openshift-network-operator/iptables-alerter-bq4dz"
Sep 11 13:04:39.279251 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.279208    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twxkm\" (UniqueName: \"kubernetes.io/projected/c8320a6e-2b9f-4253-8549-bb485d077d99-kube-api-access-twxkm\") pod \"multus-l6vms\" (UID: \"c8320a6e-2b9f-4253-8549-bb485d077d99\") " pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.288635 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.288598    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:39.288635 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.288635    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:39.288862 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.288650    2554 projected.go:196] Error preparing data for projected volume kube-api-access-8ntnk for pod openshift-network-diagnostics/network-check-target-qbm58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:39.288862 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.288817    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk podName:439302b6-1aa6-4530-9e4f-3534c305b8e1 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:39.788793155 +0000 UTC m=+3.106250193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8ntnk" (UniqueName: "kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk") pod "network-check-target-qbm58" (UID: "439302b6-1aa6-4530-9e4f-3534c305b8e1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:39.291543 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.291515    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98ghk\" (UniqueName: \"kubernetes.io/projected/88fb6546-390a-48f2-a61f-93bce22f0c64-kube-api-access-98ghk\") pod \"ovnkube-node-46vhh\" (UID: \"88fb6546-390a-48f2-a61f-93bce22f0c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.291884 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.291857    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgwsk\" (UniqueName: \"kubernetes.io/projected/0498fed7-8022-4a6b-9e69-3a560c5f5e02-kube-api-access-dgwsk\") pod \"network-metrics-daemon-9rsjp\" (UID: \"0498fed7-8022-4a6b-9e69-3a560c5f5e02\") " pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:39.292811 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.292790    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrfq9\" (UniqueName: \"kubernetes.io/projected/646e971b-8337-4e59-a5fa-f01eed3c8e33-kube-api-access-wrfq9\") pod \"tuned-nlfvh\" (UID: \"646e971b-8337-4e59-a5fa-f01eed3c8e33\") " pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.351079 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.351048    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt"
Sep 11 13:04:39.356694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.356663    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-twqp9"
Sep 11 13:04:39.365607 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.365576    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x89jp"
Sep 11 13:04:39.371656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.371636    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gt6zk"
Sep 11 13:04:39.379351 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.379329    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l6vms"
Sep 11 13:04:39.387793 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.387769    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8hpxh"
Sep 11 13:04:39.395396 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.395354    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nlfvh"
Sep 11 13:04:39.402154 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.402131    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bq4dz"
Sep 11 13:04:39.407993 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.407969    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:04:39.666231 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.666147    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:39.666417 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.666290    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:39.666417 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.666358    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret podName:c6942647-056e-4b32-a2b4-ea2e90f7be76 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:40.666341673 +0000 UTC m=+3.983798704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret") pod "global-pull-secret-syncer-p5hww" (UID: "c6942647-056e-4b32-a2b4-ea2e90f7be76") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:39.767412 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.767355    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs\") pod \"network-metrics-daemon-9rsjp\" (UID: \"0498fed7-8022-4a6b-9e69-3a560c5f5e02\") " pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:39.767630 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.767523    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:39.767630 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.767598    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs podName:0498fed7-8022-4a6b-9e69-3a560c5f5e02 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:40.767578849 +0000 UTC m=+4.085035872 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs") pod "network-metrics-daemon-9rsjp" (UID: "0498fed7-8022-4a6b-9e69-3a560c5f5e02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:39.868148 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:39.868111    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ntnk\" (UniqueName: \"kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk\") pod \"network-check-target-qbm58\" (UID: \"439302b6-1aa6-4530-9e4f-3534c305b8e1\") " pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:39.868325 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.868257    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:39.868325 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.868272    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:39.868325 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.868281    2554 projected.go:196] Error preparing data for projected volume kube-api-access-8ntnk for pod openshift-network-diagnostics/network-check-target-qbm58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:39.868500 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:39.868335    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk podName:439302b6-1aa6-4530-9e4f-3534c305b8e1 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:40.868322027 +0000 UTC m=+4.185779058 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ntnk" (UniqueName: "kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk") pod "network-check-target-qbm58" (UID: "439302b6-1aa6-4530-9e4f-3534c305b8e1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:40.009502 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.009423    2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:40.065319 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:40.065294    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88fb6546_390a_48f2_a61f_93bce22f0c64.slice/crio-b9dbdc1105a775b469cacd961595f31db4146dc8e73334b5a93d147f8433c167 WatchSource:0}: Error finding container b9dbdc1105a775b469cacd961595f31db4146dc8e73334b5a93d147f8433c167: Status 404 returned error can't find the container with id b9dbdc1105a775b469cacd961595f31db4146dc8e73334b5a93d147f8433c167
Sep 11 13:04:40.066772 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:40.066649    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8320a6e_2b9f_4253_8549_bb485d077d99.slice/crio-304c6faafcdea5ea6ab9612a27f068464dd22b41824dbf5ee45680131b0f8030 WatchSource:0}: Error finding container 304c6faafcdea5ea6ab9612a27f068464dd22b41824dbf5ee45680131b0f8030: Status 404 returned error can't find the container with id 304c6faafcdea5ea6ab9612a27f068464dd22b41824dbf5ee45680131b0f8030
Sep 11 13:04:40.068615 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:40.068517    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod646e971b_8337_4e59_a5fa_f01eed3c8e33.slice/crio-c4e279dd6f9dcd32d82d16fcbbac939c9fec960dd60924d144e592e3d7b4013c WatchSource:0}: Error finding container c4e279dd6f9dcd32d82d16fcbbac939c9fec960dd60924d144e592e3d7b4013c: Status 404 returned error can't find the container with id c4e279dd6f9dcd32d82d16fcbbac939c9fec960dd60924d144e592e3d7b4013c
Sep 11 13:04:40.071124 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:40.071105    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1572be9a_6852_40f5_951c_b0a82cd9961f.slice/crio-a04f9a18c180f3ac3a52b56fe3f04beef6b54c596fa1b9d872e34a7133f5524a WatchSource:0}: Error finding container a04f9a18c180f3ac3a52b56fe3f04beef6b54c596fa1b9d872e34a7133f5524a: Status 404 returned error can't find the container with id a04f9a18c180f3ac3a52b56fe3f04beef6b54c596fa1b9d872e34a7133f5524a
Sep 11 13:04:40.071741 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:40.071726    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ded15f4_81ba_411c_8cc0_1b3a31612e4e.slice/crio-3f307d56799a25f8350f16e14c3a5c93b80294dc6ea50182ea57ec8c2b8a7b75 WatchSource:0}: Error finding container 3f307d56799a25f8350f16e14c3a5c93b80294dc6ea50182ea57ec8c2b8a7b75: Status 404 returned error can't find the container with id 3f307d56799a25f8350f16e14c3a5c93b80294dc6ea50182ea57ec8c2b8a7b75
Sep 11 13:04:40.072826 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:40.072740    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0e0990_ad45_4112_823a_dad4edc125bf.slice/crio-8c1b046520836195689941366276c22b240fe19c8fbcb4283018113cb2425cc3 WatchSource:0}: Error finding container 8c1b046520836195689941366276c22b240fe19c8fbcb4283018113cb2425cc3: Status 404 returned error can't find the container with id 8c1b046520836195689941366276c22b240fe19c8fbcb4283018113cb2425cc3
Sep 11 13:04:40.073634 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:04:40.073607    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0312884_f43d_4476_8501_83abe961b238.slice/crio-85ee292097e4b43d7c0359eb507e74bd4ab143d6c8ed74cd283d5c39d88bb18c WatchSource:0}: Error finding container 85ee292097e4b43d7c0359eb507e74bd4ab143d6c8ed74cd283d5c39d88bb18c: Status 404 returned error can't find the container with id 85ee292097e4b43d7c0359eb507e74bd4ab143d6c8ed74cd283d5c39d88bb18c
Sep 11 13:04:40.148012 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.147989    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:40.148102 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.148029    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:40.148147 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:40.148100    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:04:40.148187 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:40.148172    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:04:40.151047 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.151017    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 12:59:39 +0000 UTC" deadline="2028-05-07 22:00:24.537634658 +0000 UTC"
Sep 11 13:04:40.151047 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.151047    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14504h55m44.386592388s"
Sep 11 13:04:40.159269 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.159240    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nlfvh" event={"ID":"646e971b-8337-4e59-a5fa-f01eed3c8e33","Type":"ContainerStarted","Data":"c4e279dd6f9dcd32d82d16fcbbac939c9fec960dd60924d144e592e3d7b4013c"}
Sep 11 13:04:40.160682 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.160658    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh" event={"ID":"88fb6546-390a-48f2-a61f-93bce22f0c64","Type":"ContainerStarted","Data":"b9dbdc1105a775b469cacd961595f31db4146dc8e73334b5a93d147f8433c167"}
Sep 11 13:04:40.163277 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.163254    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal" event={"ID":"9c13e4afb5118b6c262250604375f150","Type":"ContainerStarted","Data":"8bbca2ef3ebab3dd5a50e138175be6f3a1fbcf0ae9bd82a52fb5eb93e231e348"}
Sep 11 13:04:40.164434 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.164388    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-twqp9" event={"ID":"3b49a550-e226-4de1-8351-a9fd45b4f76f","Type":"ContainerStarted","Data":"d6b795311e3dc4e20efbc7a572089990db16987ac3e2abf3fbecc981d7da5732"}
Sep 11 13:04:40.165663 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.165640    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bq4dz" event={"ID":"7d0e0990-ad45-4112-823a-dad4edc125bf","Type":"ContainerStarted","Data":"8c1b046520836195689941366276c22b240fe19c8fbcb4283018113cb2425cc3"}
Sep 11 13:04:40.167011 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.166990    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt" event={"ID":"5ded15f4-81ba-411c-8cc0-1b3a31612e4e","Type":"ContainerStarted","Data":"3f307d56799a25f8350f16e14c3a5c93b80294dc6ea50182ea57ec8c2b8a7b75"}
Sep 11 13:04:40.168165 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.168144    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x89jp" event={"ID":"1572be9a-6852-40f5-951c-b0a82cd9961f","Type":"ContainerStarted","Data":"a04f9a18c180f3ac3a52b56fe3f04beef6b54c596fa1b9d872e34a7133f5524a"}
Sep 11 13:04:40.169357 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.169338    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l6vms" event={"ID":"c8320a6e-2b9f-4253-8549-bb485d077d99","Type":"ContainerStarted","Data":"304c6faafcdea5ea6ab9612a27f068464dd22b41824dbf5ee45680131b0f8030"}
Sep 11 13:04:40.170482 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.170461    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8hpxh" event={"ID":"00d20a97-d950-479a-9e5d-0ebb126e3e9c","Type":"ContainerStarted","Data":"d837e0909241cb973455dc5e3f46abcff5ac4bc89c4b2426a6fef02105b632e3"}
Sep 11 13:04:40.171568 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.171539    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gt6zk" event={"ID":"b0312884-f43d-4476-8501-83abe961b238","Type":"ContainerStarted","Data":"85ee292097e4b43d7c0359eb507e74bd4ab143d6c8ed74cd283d5c39d88bb18c"}
Sep 11 13:04:40.207423 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.207343    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal" podStartSLOduration=2.207320584 podStartE2EDuration="2.207320584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:40.207231044 +0000 UTC m=+3.524688084" watchObservedRunningTime="2026-09-11 13:04:40.207320584 +0000 UTC m=+3.524777645"
Sep 11 13:04:40.674425 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.673734    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:40.674425 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:40.673943    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:40.674425 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:40.674009    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret podName:c6942647-056e-4b32-a2b4-ea2e90f7be76 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:42.673991332 +0000 UTC m=+5.991448366 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret") pod "global-pull-secret-syncer-p5hww" (UID: "c6942647-056e-4b32-a2b4-ea2e90f7be76") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:40.774395 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.774342    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs\") pod \"network-metrics-daemon-9rsjp\" (UID: \"0498fed7-8022-4a6b-9e69-3a560c5f5e02\") " pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:40.774606 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:40.774587    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:40.774700 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:40.774660    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs podName:0498fed7-8022-4a6b-9e69-3a560c5f5e02 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:42.774640933 +0000 UTC m=+6.092097967 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs") pod "network-metrics-daemon-9rsjp" (UID: "0498fed7-8022-4a6b-9e69-3a560c5f5e02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:40.874842 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:40.874794    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ntnk\" (UniqueName: \"kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk\") pod \"network-check-target-qbm58\" (UID: \"439302b6-1aa6-4530-9e4f-3534c305b8e1\") " pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:40.875040 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:40.875019    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:40.875124 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:40.875046    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:40.875124 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:40.875058    2554 projected.go:196] Error preparing data for projected volume kube-api-access-8ntnk for pod openshift-network-diagnostics/network-check-target-qbm58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:40.875224 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:40.875119    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk podName:439302b6-1aa6-4530-9e4f-3534c305b8e1 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:42.875099623 +0000 UTC m=+6.192556660 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ntnk" (UniqueName: "kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk") pod "network-check-target-qbm58" (UID: "439302b6-1aa6-4530-9e4f-3534c305b8e1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:41.152176 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:41.152101    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:41.152907 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:41.152274    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:04:41.152907 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:41.152808    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 12:59:39 +0000 UTC" deadline="2028-06-21 03:57:00.367831026 +0000 UTC"
Sep 11 13:04:41.152907 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:41.152853    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15566h52m19.214981391s"
Sep 11 13:04:41.194562 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:41.194521    2554 generic.go:356] "Generic (PLEG): container finished" podID="7bf4c3f23d059c008c3e22fafba0aab7" containerID="06709e5da754f94fe0be921840d9d5a18824db22f355e726e34374c0c29f70dc" exitCode=0
Sep 11 13:04:41.195586 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:41.195549    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal" event={"ID":"7bf4c3f23d059c008c3e22fafba0aab7","Type":"ContainerDied","Data":"06709e5da754f94fe0be921840d9d5a18824db22f355e726e34374c0c29f70dc"}
Sep 11 13:04:42.147999 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:42.147285    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:42.147999 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:42.147454    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:04:42.147999 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:42.147852    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:42.147999 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:42.147948    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:04:42.227960 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:42.227876    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal" event={"ID":"7bf4c3f23d059c008c3e22fafba0aab7","Type":"ContainerStarted","Data":"9cbba0d2366e3b18bb2f5edc6fe8ca7a069e203f04477a11bd2b99a1fffdab7f"}
Sep 11 13:04:42.693156 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:42.693063    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:42.693336 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:42.693206    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:42.693336 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:42.693282    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret podName:c6942647-056e-4b32-a2b4-ea2e90f7be76 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:46.693262546 +0000 UTC m=+10.010719578 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret") pod "global-pull-secret-syncer-p5hww" (UID: "c6942647-056e-4b32-a2b4-ea2e90f7be76") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:42.793913 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:42.793874    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs\") pod \"network-metrics-daemon-9rsjp\" (UID: \"0498fed7-8022-4a6b-9e69-3a560c5f5e02\") " pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:42.794101 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:42.794040    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:42.794168 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:42.794106    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs podName:0498fed7-8022-4a6b-9e69-3a560c5f5e02 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:46.794087276 +0000 UTC m=+10.111544308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs") pod "network-metrics-daemon-9rsjp" (UID: "0498fed7-8022-4a6b-9e69-3a560c5f5e02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:42.894451 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:42.894399    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ntnk\" (UniqueName: \"kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk\") pod \"network-check-target-qbm58\" (UID: \"439302b6-1aa6-4530-9e4f-3534c305b8e1\") " pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:42.894637 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:42.894596    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:42.894637 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:42.894616    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:42.894637 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:42.894629    2554 projected.go:196] Error preparing data for projected volume kube-api-access-8ntnk for pod openshift-network-diagnostics/network-check-target-qbm58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:42.894851 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:42.894688    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk podName:439302b6-1aa6-4530-9e4f-3534c305b8e1 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:46.89466946 +0000 UTC m=+10.212126494 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ntnk" (UniqueName: "kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk") pod "network-check-target-qbm58" (UID: "439302b6-1aa6-4530-9e4f-3534c305b8e1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:43.149835 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:43.147490    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:43.150533 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:43.150157    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:04:44.148040 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:44.148003    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:44.148545 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:44.148178    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:04:44.148619 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:44.148601    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:44.148791 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:44.148701    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:04:45.150778 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:45.150739    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:45.151259 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:45.150834    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:04:46.147694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:46.147509    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:46.151337 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:46.148890    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:04:46.151337 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:46.148955    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:46.151337 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:46.149201    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:04:46.729912 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:46.729249    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:46.729912 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:46.729448    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:46.729912 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:46.729514    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret podName:c6942647-056e-4b32-a2b4-ea2e90f7be76 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:54.729496149 +0000 UTC m=+18.046953190 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret") pod "global-pull-secret-syncer-p5hww" (UID: "c6942647-056e-4b32-a2b4-ea2e90f7be76") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:46.830706 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:46.830659    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs\") pod \"network-metrics-daemon-9rsjp\" (UID: \"0498fed7-8022-4a6b-9e69-3a560c5f5e02\") " pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:46.830889 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:46.830840    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:46.830970 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:46.830918    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs podName:0498fed7-8022-4a6b-9e69-3a560c5f5e02 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:54.830896238 +0000 UTC m=+18.148353279 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs") pod "network-metrics-daemon-9rsjp" (UID: "0498fed7-8022-4a6b-9e69-3a560c5f5e02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:46.931248 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:46.931170    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ntnk\" (UniqueName: \"kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk\") pod \"network-check-target-qbm58\" (UID: \"439302b6-1aa6-4530-9e4f-3534c305b8e1\") " pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:46.931430 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:46.931391    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:46.931430 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:46.931419    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:46.931527 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:46.931433    2554 projected.go:196] Error preparing data for projected volume kube-api-access-8ntnk for pod openshift-network-diagnostics/network-check-target-qbm58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:46.931527 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:46.931500    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk podName:439302b6-1aa6-4530-9e4f-3534c305b8e1 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:54.931481857 +0000 UTC m=+18.248938888 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ntnk" (UniqueName: "kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk") pod "network-check-target-qbm58" (UID: "439302b6-1aa6-4530-9e4f-3534c305b8e1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:47.148682 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:47.148142    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:47.148682 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:47.148263    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:04:48.147732 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:48.147698    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:48.148192 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:48.147846    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:04:48.148261 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:48.148228    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:48.148343 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:48.148321    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:04:49.147826 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:49.147786    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:49.148306 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:49.147931    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:04:50.147780 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:50.147740    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:50.147780 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:50.147767    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:50.148229 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:50.147867    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:04:50.148229 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:50.148009    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:04:51.148025 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:51.147986    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:51.148548 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:51.148128    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:04:52.148154 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:52.148110    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:52.148668 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:52.148129    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:52.148668 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:52.148262    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:04:52.148668 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:52.148374    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:04:53.151059 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:53.151028    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:53.151492 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:53.151126    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:04:54.147831 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:54.147795    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:54.148028 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:54.147795    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:54.148028 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:54.147907    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:04:54.148028 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:54.147981    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:04:54.786555 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:54.786520    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:54.786988 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:54.786703    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:54.786988 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:54.786766    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret podName:c6942647-056e-4b32-a2b4-ea2e90f7be76 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:10.786749927 +0000 UTC m=+34.104206972 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret") pod "global-pull-secret-syncer-p5hww" (UID: "c6942647-056e-4b32-a2b4-ea2e90f7be76") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:54.887531 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:54.887489    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs\") pod \"network-metrics-daemon-9rsjp\" (UID: \"0498fed7-8022-4a6b-9e69-3a560c5f5e02\") " pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:54.887723 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:54.887661    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:54.887782 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:54.887741    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs podName:0498fed7-8022-4a6b-9e69-3a560c5f5e02 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:10.887721075 +0000 UTC m=+34.205178108 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs") pod "network-metrics-daemon-9rsjp" (UID: "0498fed7-8022-4a6b-9e69-3a560c5f5e02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:54.988878 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:54.988837    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ntnk\" (UniqueName: \"kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk\") pod \"network-check-target-qbm58\" (UID: \"439302b6-1aa6-4530-9e4f-3534c305b8e1\") " pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:54.989043 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:54.989015    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:54.989085 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:54.989043    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:54.989085 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:54.989060    2554 projected.go:196] Error preparing data for projected volume kube-api-access-8ntnk for pod openshift-network-diagnostics/network-check-target-qbm58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:54.989182 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:54.989120    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk podName:439302b6-1aa6-4530-9e4f-3534c305b8e1 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:10.98910154 +0000 UTC m=+34.306558572 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ntnk" (UniqueName: "kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk") pod "network-check-target-qbm58" (UID: "439302b6-1aa6-4530-9e4f-3534c305b8e1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:55.147489 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:55.147402    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:55.147654 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:55.147535    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:04:56.148171 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:56.148125    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:56.148171 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:56.148166    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:56.148724 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:56.148274    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:04:56.148724 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:56.148435    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:04:57.148476 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:57.148441    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:57.148864 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:57.148546    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:04:58.147527 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:58.147086    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:04:58.147527 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:58.147225    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:04:58.147527 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:58.147289    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:04:58.147527 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:58.147401    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:04:58.271287 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:58.271253    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8hpxh" event={"ID":"00d20a97-d950-479a-9e5d-0ebb126e3e9c","Type":"ContainerStarted","Data":"cb3efdfad2bd62519e4cf431a548248cdfecc0eb949425cc9229dcd8f460985d"}
Sep 11 13:04:58.300247 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:58.297638    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal" podStartSLOduration=20.297617217 podStartE2EDuration="20.297617217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:42.257985111 +0000 UTC m=+5.575442153" watchObservedRunningTime="2026-09-11 13:04:58.297617217 +0000 UTC m=+21.615074258"
Sep 11 13:04:59.147873 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.147630    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:04:59.148074 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:04:59.147984    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:04:59.275466 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.275434    2554 generic.go:356] "Generic (PLEG): container finished" podID="b0312884-f43d-4476-8501-83abe961b238" containerID="80971010bb668dcf88b39e0b93a39dd50cacb17a3540c4fce92d38906d003977" exitCode=0
Sep 11 13:04:59.276537 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.275520    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gt6zk" event={"ID":"b0312884-f43d-4476-8501-83abe961b238","Type":"ContainerDied","Data":"80971010bb668dcf88b39e0b93a39dd50cacb17a3540c4fce92d38906d003977"}
Sep 11 13:04:59.277469 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.277189    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nlfvh" event={"ID":"646e971b-8337-4e59-a5fa-f01eed3c8e33","Type":"ContainerStarted","Data":"21de2792628347d8c91ceca32891a897eebf9f5584ad2013ae4c275c82ddd1a6"}
Sep 11 13:04:59.280323 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.280304    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh" event={"ID":"88fb6546-390a-48f2-a61f-93bce22f0c64","Type":"ContainerStarted","Data":"a58e1793436e61d09b908fb300df18c9c834527577b86ddf088eb7d4c5e0f172"}
Sep 11 13:04:59.280432 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.280377    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh" event={"ID":"88fb6546-390a-48f2-a61f-93bce22f0c64","Type":"ContainerStarted","Data":"e67583f0e68ac0bb101316a9a3c464821f35a164a9d0ae23502a15eb5cbf04aa"}
Sep 11 13:04:59.280432 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.280412    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh" event={"ID":"88fb6546-390a-48f2-a61f-93bce22f0c64","Type":"ContainerStarted","Data":"48512f2ec90c31c7da0c10998632f04d8d6feccec22ecc7966141f7e384a07f8"}
Sep 11 13:04:59.280515 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.280440    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh" event={"ID":"88fb6546-390a-48f2-a61f-93bce22f0c64","Type":"ContainerStarted","Data":"361c664d6d78fdbff906c5d9442e34fab47da88e7131fc6367a143e9733defe9"}
Sep 11 13:04:59.280515 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.280476    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh" event={"ID":"88fb6546-390a-48f2-a61f-93bce22f0c64","Type":"ContainerStarted","Data":"97f0bd2c767703ace4c3ab99071e2444a7e74db5c40413e4699ef505ddc5af9a"}
Sep 11 13:04:59.280515 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.280505    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh" event={"ID":"88fb6546-390a-48f2-a61f-93bce22f0c64","Type":"ContainerStarted","Data":"7f67a129fef4daa97047965cd44d542425aa4b08c3d1476d598032504e38597c"}
Sep 11 13:04:59.281648 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.281627    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-twqp9" event={"ID":"3b49a550-e226-4de1-8351-a9fd45b4f76f","Type":"ContainerStarted","Data":"02a79745ce1ed929890593096032e63f1ddc60c3c6999d0d60aee99999ad6fac"}
Sep 11 13:04:59.283459 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.283436    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt" event={"ID":"5ded15f4-81ba-411c-8cc0-1b3a31612e4e","Type":"ContainerStarted","Data":"919f2aa0e672373f5340a84ba09e78040cafd28e2fe179423d25b55698652e1f"}
Sep 11 13:04:59.284848 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.284825    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x89jp" event={"ID":"1572be9a-6852-40f5-951c-b0a82cd9961f","Type":"ContainerStarted","Data":"6245ca542c396be98a886a0e9e7d9301ac786c6459e4d860a6eebaac03202a3d"}
Sep 11 13:04:59.286253 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.286234    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l6vms" event={"ID":"c8320a6e-2b9f-4253-8549-bb485d077d99","Type":"ContainerStarted","Data":"f2047fa1cd2e2ddb91520f6562119a95114af76a22e393a1d2e5d107de5d54a8"}
Sep 11 13:04:59.309429 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.309379    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8hpxh" podStartSLOduration=11.662076265 podStartE2EDuration="22.309346445s" totalImagesPullingTime="10.64727018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:58.296655857 +0000 UTC m=+21.614112898" watchObservedRunningTime="2026-09-11 13:04:59.309346445 +0000 UTC m=+22.626803561"
Sep 11 13:04:59.364723 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.364667    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nlfvh" podStartSLOduration=4.311502643 podStartE2EDuration="22.364650714s" totalImagesPullingTime="18.053148071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:59.364589607 +0000 UTC m=+22.682046647" watchObservedRunningTime="2026-09-11 13:04:59.364650714 +0000 UTC m=+22.682107748"
Sep 11 13:04:59.365042 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.365021    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-l6vms" podStartSLOduration=4.2512732060000005 podStartE2EDuration="22.365015878s" totalImagesPullingTime="18.113742672s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:59.333253322 +0000 UTC m=+22.650710362" watchObservedRunningTime="2026-09-11 13:04:59.365015878 +0000 UTC m=+22.682472969"
Sep 11 13:04:59.394927 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.394804    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-x89jp" podStartSLOduration=4.356989868 podStartE2EDuration="22.394788106s" totalImagesPullingTime="18.037798238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:59.394300897 +0000 UTC m=+22.711757936" watchObservedRunningTime="2026-09-11 13:04:59.394788106 +0000 UTC m=+22.712245146"
Sep 11 13:04:59.713767 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:04:59.713729    2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 13:05:00.022476 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:00.022358    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8hpxh"
Sep 11 13:05:00.022974 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:00.022957    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8hpxh"
Sep 11 13:05:00.046609 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:00.046549    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-twqp9" podStartSLOduration=5.013750902 podStartE2EDuration="23.046531965s" totalImagesPullingTime="18.032781063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:59.415837522 +0000 UTC m=+22.733294587" watchObservedRunningTime="2026-09-11 13:05:00.046531965 +0000 UTC m=+23.363989006"
Sep 11 13:05:00.111451 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:00.111300    2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T13:04:59.713763274Z","UUID":"5a0a6756-2641-4010-afa1-ea6d48537a54","Handler":null,"Name":"","Endpoint":""}
Sep 11 13:05:00.113017 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:00.112997    2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 13:05:00.113111 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:00.113029    2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 13:05:00.147443 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:00.147407    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:05:00.147443 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:00.147446    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:05:00.147700 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:00.147546    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:05:00.147700 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:00.147644    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:05:00.290439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:00.290338    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bq4dz" event={"ID":"7d0e0990-ad45-4112-823a-dad4edc125bf","Type":"ContainerStarted","Data":"8ebe7c0ed9f2ab1deeeeeee5690963714c393848969651ea0c54536cf0d9cc60"}
Sep 11 13:05:00.292617 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:00.292583    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt" event={"ID":"5ded15f4-81ba-411c-8cc0-1b3a31612e4e","Type":"ContainerStarted","Data":"d73bd34fc2673848c55dcc9c124d2e0a8037754feabca4d25ce52485aa7de620"}
Sep 11 13:05:00.313429 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:00.313377    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bq4dz" podStartSLOduration=5.27930256 podStartE2EDuration="23.313342627s" totalImagesPullingTime="18.034040067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:00.312751929 +0000 UTC m=+23.630208963" watchObservedRunningTime="2026-09-11 13:05:00.313342627 +0000 UTC m=+23.630799668"
Sep 11 13:05:01.148110 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:01.148026    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:05:01.148331 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:01.148157    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:05:01.298472 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:01.298430    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh" event={"ID":"88fb6546-390a-48f2-a61f-93bce22f0c64","Type":"ContainerStarted","Data":"c2d52f131b3074274f3d4a89e1d0d9a9627caceb78460e75c414c61b8f5d46c9"}
Sep 11 13:05:01.300660 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:01.300628    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt" event={"ID":"5ded15f4-81ba-411c-8cc0-1b3a31612e4e","Type":"ContainerStarted","Data":"140b921995fc10789b995c7c24a801765daba3146a8acbf94677bad9918eda9d"}
Sep 11 13:05:01.301025 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:01.301007    2554 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 13:05:01.322449 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:01.322386    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxhlt" podStartSLOduration=4.127607267 podStartE2EDuration="24.322349201s" totalImagesPullingTime="20.194741934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:01.322023399 +0000 UTC m=+24.639480438" watchObservedRunningTime="2026-09-11 13:05:01.322349201 +0000 UTC m=+24.639806240"
Sep 11 13:05:02.147574 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:02.147521    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:05:02.147752 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:02.147522    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:05:02.147752 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:02.147635    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:05:02.147752 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:02.147713    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:05:03.147759 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:03.147672    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:05:03.148145 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:03.147800    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:05:04.147549 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:04.147517    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:05:04.147712 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:04.147561    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:05:04.147712 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:04.147648    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:05:04.148429 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:04.147754    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:05:04.308810 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:04.308784    2554 generic.go:356] "Generic (PLEG): container finished" podID="b0312884-f43d-4476-8501-83abe961b238" containerID="16b9d4239b0ebf6ac756a7f19ec049ef7c67fbd68eb437710a84be0b001c882c" exitCode=0
Sep 11 13:05:04.308897 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:04.308862    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gt6zk" event={"ID":"b0312884-f43d-4476-8501-83abe961b238","Type":"ContainerDied","Data":"16b9d4239b0ebf6ac756a7f19ec049ef7c67fbd68eb437710a84be0b001c882c"}
Sep 11 13:05:04.313422 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:04.313399    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh" event={"ID":"88fb6546-390a-48f2-a61f-93bce22f0c64","Type":"ContainerStarted","Data":"ca48d3f659097f6448ef1143b46d706ed24a1e166d544588d1f894ce7b133b24"}
Sep 11 13:05:04.313668 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:04.313655    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:05:04.328514 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:04.328464    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:05:04.382426 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:04.382348    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh" podStartSLOduration=9.325359175 podStartE2EDuration="27.38233166s" totalImagesPullingTime="18.056972485s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:04.382225803 +0000 UTC m=+27.699682873" watchObservedRunningTime="2026-09-11 13:05:04.38233166 +0000 UTC m=+27.699788699"
Sep 11 13:05:05.148352 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:05.148100    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:05:05.148816 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:05.148465    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:05:05.316341 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:05.316313    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:05:05.316495 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:05.316414    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:05:05.330885 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:05.330857    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:05:05.778090 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:05.778063    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9rsjp"]
Sep 11 13:05:05.778248 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:05.778194    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:05:05.778337 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:05.778314    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:05:05.781735 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:05.781712    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qbm58"]
Sep 11 13:05:05.781833 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:05.781788    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:05:05.781890 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:05.781868    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:05:05.801259 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:05.801232    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p5hww"]
Sep 11 13:05:05.801418 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:05.801333    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:05:05.801481 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:05.801433    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:05:06.319948 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:06.319916    2554 generic.go:356] "Generic (PLEG): container finished" podID="b0312884-f43d-4476-8501-83abe961b238" containerID="45e746a4ad9bce64462634c3fcfa2d47addc9a63a5b2e157f609ff167701e735" exitCode=0
Sep 11 13:05:06.320330 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:06.320000    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gt6zk" event={"ID":"b0312884-f43d-4476-8501-83abe961b238","Type":"ContainerDied","Data":"45e746a4ad9bce64462634c3fcfa2d47addc9a63a5b2e157f609ff167701e735"}
Sep 11 13:05:06.906677 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:06.906643    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8hpxh"
Sep 11 13:05:06.906844 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:06.906807    2554 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 13:05:06.907306 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:06.907282    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8hpxh"
Sep 11 13:05:07.149523 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:07.149498    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:05:07.149904 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:07.149878    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:05:07.150001 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:07.149769    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:05:07.150161 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:07.150143    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:05:07.150256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:07.149739    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:05:07.151192 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:07.151167    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:05:08.327041 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:08.326790    2554 generic.go:356] "Generic (PLEG): container finished" podID="b0312884-f43d-4476-8501-83abe961b238" containerID="35dbe3b270cec9b9480692dcb91c6a91c3a981bb2ebce4c0bb6def104a323105" exitCode=0
Sep 11 13:05:08.327430 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:08.326869    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gt6zk" event={"ID":"b0312884-f43d-4476-8501-83abe961b238","Type":"ContainerDied","Data":"35dbe3b270cec9b9480692dcb91c6a91c3a981bb2ebce4c0bb6def104a323105"}
Sep 11 13:05:09.148160 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:09.148118    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:05:09.148326 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:09.148118    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:05:09.148326 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:09.148255    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p5hww" podUID="c6942647-056e-4b32-a2b4-ea2e90f7be76"
Sep 11 13:05:09.148326 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:09.148133    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:05:09.148499 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:09.148353    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbm58" podUID="439302b6-1aa6-4530-9e4f-3534c305b8e1"
Sep 11 13:05:09.148499 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:09.148453    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9rsjp" podUID="0498fed7-8022-4a6b-9e69-3a560c5f5e02"
Sep 11 13:05:10.802048 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:10.802017    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:05:10.802666 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:10.802194    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:05:10.802666 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:10.802268    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret podName:c6942647-056e-4b32-a2b4-ea2e90f7be76 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:42.802248735 +0000 UTC m=+66.119705769 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret") pod "global-pull-secret-syncer-p5hww" (UID: "c6942647-056e-4b32-a2b4-ea2e90f7be76") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:05:10.903284 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:10.903249    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs\") pod \"network-metrics-daemon-9rsjp\" (UID: \"0498fed7-8022-4a6b-9e69-3a560c5f5e02\") " pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:05:10.903482 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:10.903453    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:05:10.903544 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:10.903533    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs podName:0498fed7-8022-4a6b-9e69-3a560c5f5e02 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:42.903512135 +0000 UTC m=+66.220969157 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs") pod "network-metrics-daemon-9rsjp" (UID: "0498fed7-8022-4a6b-9e69-3a560c5f5e02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:05:10.986483 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:10.986448    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeReady"
Sep 11 13:05:10.986649 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:10.986603    2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 13:05:11.003802 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.003770    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ntnk\" (UniqueName: \"kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk\") pod \"network-check-target-qbm58\" (UID: \"439302b6-1aa6-4530-9e4f-3534c305b8e1\") " pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:05:11.004008 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.003935    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:05:11.004008 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.003956    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:05:11.004008 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.003970    2554 projected.go:196] Error preparing data for projected volume kube-api-access-8ntnk for pod openshift-network-diagnostics/network-check-target-qbm58: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:05:11.004179 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.004025    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk podName:439302b6-1aa6-4530-9e4f-3534c305b8e1 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:43.00400851 +0000 UTC m=+66.321465530 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ntnk" (UniqueName: "kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk") pod "network-check-target-qbm58" (UID: "439302b6-1aa6-4530-9e4f-3534c305b8e1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:05:11.036252 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.036212    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"]
Sep 11 13:05:11.038913 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.038883    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jfl55"]
Sep 11 13:05:11.039786 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.039017    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:11.040475 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.040454    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6548b9b645-mln9l"]
Sep 11 13:05:11.040607 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.040579    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jfl55"
Sep 11 13:05:11.042025 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.042007    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-p49pp"]
Sep 11 13:05:11.042132 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.042105    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.043568 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.043549    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"]
Sep 11 13:05:11.043665 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.043639    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.044757 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.044734    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 11 13:05:11.046973 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.045538    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-crft9\""
Sep 11 13:05:11.046973 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.045740    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 13:05:11.046973 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.045940    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 11 13:05:11.046973 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.045538    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 11 13:05:11.046973 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.046354    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 13:05:11.046973 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.046476    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6b946bb9b8-kxhmt"]
Sep 11 13:05:11.046973 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.046701    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:05:11.048058 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.047659    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"
Sep 11 13:05:11.048058 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.047795    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 13:05:11.048058 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.048000    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 13:05:11.050125 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.048429    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zxvfq\""
Sep 11 13:05:11.050125 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.048547    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 11 13:05:11.050125 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.048719    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 13:05:11.050125 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.049423    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 13:05:11.050125 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.049762    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rw5gp\""
Sep 11 13:05:11.050125 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.049870    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-n5c4g"]
Sep 11 13:05:11.050736 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.050205    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.051024 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.051004    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 11 13:05:11.052684 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.052664    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 13:05:11.053003 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.052985    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-6jwmf\""
Sep 11 13:05:11.054528 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.054508    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.056067 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.056037    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:05:11.056237 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.056225    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 11 13:05:11.056630 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.056511    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 13:05:11.057699 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.057681    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 11 13:05:11.057899 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.057870    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-t87jn\""
Sep 11 13:05:11.057979 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.057778    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 11 13:05:11.058067 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.058049    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 13:05:11.058266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.058251    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 11 13:05:11.058495 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.058469    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 11 13:05:11.058575 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.058567    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 13:05:11.059962 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.059935    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 11 13:05:11.060175 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.060151    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tffbk\""
Sep 11 13:05:11.060266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.060199    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-k8722\""
Sep 11 13:05:11.060457 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.060440    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 11 13:05:11.060645 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.060627    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 11 13:05:11.060745 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.060498    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 11 13:05:11.061263 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.061009    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:05:11.061263 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.061227    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 11 13:05:11.062913 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.062892    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jfl55"]
Sep 11 13:05:11.063509 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.063486    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"]
Sep 11 13:05:11.065843 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.064159    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-p49pp"]
Sep 11 13:05:11.068634 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.068166    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-n5c4g"]
Sep 11 13:05:11.068989 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.068878    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6b946bb9b8-kxhmt"]
Sep 11 13:05:11.070559 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.070535    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 11 13:05:11.075544 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.075527    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"]
Sep 11 13:05:11.078001 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.077982    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"
Sep 11 13:05:11.078215 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.078197    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-l7wdt"]
Sep 11 13:05:11.080227 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.080172    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 11 13:05:11.080492 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.080475    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l7wdt"
Sep 11 13:05:11.080819 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.080684    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 11 13:05:11.080819 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.080741    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:05:11.081144 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.081123    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-cssw8\""
Sep 11 13:05:11.081249 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.081209    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 11 13:05:11.085667 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.085642    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 13:05:11.086069 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.086051    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4hb2m"]
Sep 11 13:05:11.087804 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.087725    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 13:05:11.087980 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.087731    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 13:05:11.088136 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.087893    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rl798\""
Sep 11 13:05:11.090630 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.090610    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"]
Sep 11 13:05:11.090713 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.090652    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6548b9b645-mln9l"]
Sep 11 13:05:11.090768 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.090751    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:11.092390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.092356    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"]
Sep 11 13:05:11.094539 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.094503    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 13:05:11.094823 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.094807    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 13:05:11.095146 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.095127    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7xqq5\""
Sep 11 13:05:11.096129 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.096106    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l7wdt"]
Sep 11 13:05:11.104253 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.104224    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.104343 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.104260    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-serving-cert\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.104343 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.104286    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-bound-sa-token\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.104493 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.104337    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw7qj\" (UniqueName: \"kubernetes.io/projected/aa53386d-9f0f-4d4e-868f-520ca3f17cde-kube-api-access-jw7qj\") pod \"volume-data-source-validator-86695b7cc7-jfl55\" (UID: \"aa53386d-9f0f-4d4e-868f-520ca3f17cde\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jfl55"
Sep 11 13:05:11.104687 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.104669    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-service-ca-bundle\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.104819 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.104716    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/458a0bb9-feca-4918-a2c4-799b4659f336-image-registry-private-configuration\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.104819 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.104775    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/458a0bb9-feca-4918-a2c4-799b4659f336-ca-trust-extracted\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.104819 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.104802    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/458a0bb9-feca-4918-a2c4-799b4659f336-registry-certificates\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.104966 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.104843    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/458a0bb9-feca-4918-a2c4-799b4659f336-trusted-ca\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.104966 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.104868    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m5b6\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-kube-api-access-4m5b6\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.104966 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.104904    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-snapshots\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.105111 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.104973    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-tmp\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.105111 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.104994    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5mnk\" (UniqueName: \"kubernetes.io/projected/6e4b7f10-74fd-40fd-ba55-61080037f297-kube-api-access-n5mnk\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:11.105111 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.105012    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6e4b7f10-74fd-40fd-ba55-61080037f297-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:11.105111 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.105070    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:11.105111 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.105106    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.105340 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.105136    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzvrm\" (UniqueName: \"kubernetes.io/projected/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-kube-api-access-jzvrm\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.105340 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.105165    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/458a0bb9-feca-4918-a2c4-799b4659f336-installation-pull-secrets\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.119628 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.119611    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4hb2m"]
Sep 11 13:05:11.147469 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.147438    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:05:11.147611 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.147435    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:05:11.147611 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.147574    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:05:11.149903 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.149880    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-np5mv\""
Sep 11 13:05:11.150034 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.149998    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:05:11.150124 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.150106    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:05:11.150217 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.150197    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lg7nk\""
Sep 11 13:05:11.150217 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.150106    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:05:11.150443 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.150339    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:05:11.206381 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206282    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbe4efbd-bb09-403a-a183-f0efd34611ee-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b65r7\" (UID: \"cbe4efbd-bb09-403a-a183-f0efd34611ee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"
Sep 11 13:05:11.206381 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206335    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzvrm\" (UniqueName: \"kubernetes.io/projected/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-kube-api-access-jzvrm\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.206381 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206354    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.206652 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206420    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-serving-cert\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.206652 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206495    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-bound-sa-token\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.206652 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206523    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:11.206652 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206556    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a975d99e-85e6-4e33-b286-149c224c5d6e-serving-cert\") pod \"console-operator-696884fb9c-n5c4g\" (UID: \"a975d99e-85e6-4e33-b286-149c224c5d6e\") " pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.206848 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206640    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/458a0bb9-feca-4918-a2c4-799b4659f336-image-registry-private-configuration\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.206848 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206681    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/458a0bb9-feca-4918-a2c4-799b4659f336-ca-trust-extracted\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.206848 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206717    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54af5720-e75b-4e36-94cc-4a72fb166eb4-tmp-dir\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:11.206848 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206766    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v5c8r\" (UID: \"32e21675-4221-4ee4-a70f-5954bb73da27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"
Sep 11 13:05:11.206848 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206792    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dj5q\" (UniqueName: \"kubernetes.io/projected/54af5720-e75b-4e36-94cc-4a72fb166eb4-kube-api-access-7dj5q\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:11.206848 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206823    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m5b6\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-kube-api-access-4m5b6\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.207133 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206876    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-snapshots\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.207133 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206912    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-stats-auth\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.207133 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206941    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-tmp\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.207133 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206964    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a975d99e-85e6-4e33-b286-149c224c5d6e-config\") pod \"console-operator-696884fb9c-n5c4g\" (UID: \"a975d99e-85e6-4e33-b286-149c224c5d6e\") " pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.207133 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.206987    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.207133 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207012    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5mnk\" (UniqueName: \"kubernetes.io/projected/6e4b7f10-74fd-40fd-ba55-61080037f297-kube-api-access-n5mnk\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:11.207133 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207039    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbe4efbd-bb09-403a-a183-f0efd34611ee-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b65r7\" (UID: \"cbe4efbd-bb09-403a-a183-f0efd34611ee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"
Sep 11 13:05:11.207133 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207073    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:11.207133 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207098    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-default-certificate\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.207133 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207133    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/458a0bb9-feca-4918-a2c4-799b4659f336-ca-trust-extracted\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.207618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207140    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/458a0bb9-feca-4918-a2c4-799b4659f336-installation-pull-secrets\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.207618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207186    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lww6p\" (UniqueName: \"kubernetes.io/projected/32e21675-4221-4ee4-a70f-5954bb73da27-kube-api-access-lww6p\") pod \"cluster-samples-operator-86499c89c4-v5c8r\" (UID: \"32e21675-4221-4ee4-a70f-5954bb73da27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"
Sep 11 13:05:11.207618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207207    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10e1239e-56fb-489b-9079-7b3c2e3619f9-cert\") pod \"ingress-canary-l7wdt\" (UID: \"10e1239e-56fb-489b-9079-7b3c2e3619f9\") " pod="openshift-ingress-canary/ingress-canary-l7wdt"
Sep 11 13:05:11.207618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207253    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw7qj\" (UniqueName: \"kubernetes.io/projected/aa53386d-9f0f-4d4e-868f-520ca3f17cde-kube-api-access-jw7qj\") pod \"volume-data-source-validator-86695b7cc7-jfl55\" (UID: \"aa53386d-9f0f-4d4e-868f-520ca3f17cde\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jfl55"
Sep 11 13:05:11.207618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207278    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-service-ca-bundle\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.207618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207299    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/458a0bb9-feca-4918-a2c4-799b4659f336-trusted-ca\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.207618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207324    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/458a0bb9-feca-4918-a2c4-799b4659f336-registry-certificates\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.207618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207382    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-tmp\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.207618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207387    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9htw5\" (UniqueName: \"kubernetes.io/projected/10e1239e-56fb-489b-9079-7b3c2e3619f9-kube-api-access-9htw5\") pod \"ingress-canary-l7wdt\" (UID: \"10e1239e-56fb-489b-9079-7b3c2e3619f9\") " pod="openshift-ingress-canary/ingress-canary-l7wdt"
Sep 11 13:05:11.207618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207454    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.207618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207454    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-snapshots\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.207618 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.207571    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:05:11.207618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207584    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54af5720-e75b-4e36-94cc-4a72fb166eb4-config-volume\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:11.208185 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.207632    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls podName:6e4b7f10-74fd-40fd-ba55-61080037f297 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:11.707614915 +0000 UTC m=+35.025071934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4rqbt" (UID: "6e4b7f10-74fd-40fd-ba55-61080037f297") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:05:11.208185 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207718    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.208185 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207765    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqr86\" (UniqueName: \"kubernetes.io/projected/a282b450-2ac0-4aed-98ac-7be36ff01407-kube-api-access-wqr86\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.208185 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207806    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a975d99e-85e6-4e33-b286-149c224c5d6e-trusted-ca\") pod \"console-operator-696884fb9c-n5c4g\" (UID: \"a975d99e-85e6-4e33-b286-149c224c5d6e\") " pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.208185 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.207841    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6e4b7f10-74fd-40fd-ba55-61080037f297-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:11.208185 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.208113    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/458a0bb9-feca-4918-a2c4-799b4659f336-registry-certificates\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.208185 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.208159    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.208511 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.208210    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9q6b\" (UniqueName: \"kubernetes.io/projected/a975d99e-85e6-4e33-b286-149c224c5d6e-kube-api-access-p9q6b\") pod \"console-operator-696884fb9c-n5c4g\" (UID: \"a975d99e-85e6-4e33-b286-149c224c5d6e\") " pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.208511 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.208242    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkskg\" (UniqueName: \"kubernetes.io/projected/cbe4efbd-bb09-403a-a183-f0efd34611ee-kube-api-access-qkskg\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b65r7\" (UID: \"cbe4efbd-bb09-403a-a183-f0efd34611ee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"
Sep 11 13:05:11.208511 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.208274    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-service-ca-bundle\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.208511 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.208296    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/458a0bb9-feca-4918-a2c4-799b4659f336-trusted-ca\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.208511 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.208349    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:05:11.208511 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.208381    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6548b9b645-mln9l: secret "image-registry-tls" not found
Sep 11 13:05:11.208511 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.208428    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls podName:458a0bb9-feca-4918-a2c4-799b4659f336 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:11.708411842 +0000 UTC m=+35.025868871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls") pod "image-registry-6548b9b645-mln9l" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336") : secret "image-registry-tls" not found
Sep 11 13:05:11.208843 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.208678    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6e4b7f10-74fd-40fd-ba55-61080037f297-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:11.211243 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.211221    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/458a0bb9-feca-4918-a2c4-799b4659f336-image-registry-private-configuration\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.211243 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.211229    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-serving-cert\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.211446 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.211296    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/458a0bb9-feca-4918-a2c4-799b4659f336-installation-pull-secrets\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.223540 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.223503    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzvrm\" (UniqueName: \"kubernetes.io/projected/7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e-kube-api-access-jzvrm\") pod \"insights-operator-5844b856db-p49pp\" (UID: \"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e\") " pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.233088 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.233063    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m5b6\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-kube-api-access-4m5b6\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.233556 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.233532    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw7qj\" (UniqueName: \"kubernetes.io/projected/aa53386d-9f0f-4d4e-868f-520ca3f17cde-kube-api-access-jw7qj\") pod \"volume-data-source-validator-86695b7cc7-jfl55\" (UID: \"aa53386d-9f0f-4d4e-868f-520ca3f17cde\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jfl55"
Sep 11 13:05:11.238848 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.238824    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-bound-sa-token\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.239048 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.239028    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5mnk\" (UniqueName: \"kubernetes.io/projected/6e4b7f10-74fd-40fd-ba55-61080037f297-kube-api-access-n5mnk\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:11.308664 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.308631    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lww6p\" (UniqueName: \"kubernetes.io/projected/32e21675-4221-4ee4-a70f-5954bb73da27-kube-api-access-lww6p\") pod \"cluster-samples-operator-86499c89c4-v5c8r\" (UID: \"32e21675-4221-4ee4-a70f-5954bb73da27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"
Sep 11 13:05:11.308664 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.308668    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10e1239e-56fb-489b-9079-7b3c2e3619f9-cert\") pod \"ingress-canary-l7wdt\" (UID: \"10e1239e-56fb-489b-9079-7b3c2e3619f9\") " pod="openshift-ingress-canary/ingress-canary-l7wdt"
Sep 11 13:05:11.308897 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.308726    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9htw5\" (UniqueName: \"kubernetes.io/projected/10e1239e-56fb-489b-9079-7b3c2e3619f9-kube-api-access-9htw5\") pod \"ingress-canary-l7wdt\" (UID: \"10e1239e-56fb-489b-9079-7b3c2e3619f9\") " pod="openshift-ingress-canary/ingress-canary-l7wdt"
Sep 11 13:05:11.308897 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.308809    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54af5720-e75b-4e36-94cc-4a72fb166eb4-config-volume\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:11.308897 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.308861    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.308897 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.308886    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqr86\" (UniqueName: \"kubernetes.io/projected/a282b450-2ac0-4aed-98ac-7be36ff01407-kube-api-access-wqr86\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.309102 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.308905    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a975d99e-85e6-4e33-b286-149c224c5d6e-trusted-ca\") pod \"console-operator-696884fb9c-n5c4g\" (UID: \"a975d99e-85e6-4e33-b286-149c224c5d6e\") " pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.309102 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.308940    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9q6b\" (UniqueName: \"kubernetes.io/projected/a975d99e-85e6-4e33-b286-149c224c5d6e-kube-api-access-p9q6b\") pod \"console-operator-696884fb9c-n5c4g\" (UID: \"a975d99e-85e6-4e33-b286-149c224c5d6e\") " pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.309102 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.308966    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkskg\" (UniqueName: \"kubernetes.io/projected/cbe4efbd-bb09-403a-a183-f0efd34611ee-kube-api-access-qkskg\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b65r7\" (UID: \"cbe4efbd-bb09-403a-a183-f0efd34611ee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"
Sep 11 13:05:11.309102 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.308991    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbe4efbd-bb09-403a-a183-f0efd34611ee-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b65r7\" (UID: \"cbe4efbd-bb09-403a-a183-f0efd34611ee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"
Sep 11 13:05:11.309102 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.309023    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:11.309102 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.309074    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a975d99e-85e6-4e33-b286-149c224c5d6e-serving-cert\") pod \"console-operator-696884fb9c-n5c4g\" (UID: \"a975d99e-85e6-4e33-b286-149c224c5d6e\") " pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.309416 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.309126    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54af5720-e75b-4e36-94cc-4a72fb166eb4-tmp-dir\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:11.309416 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.309164    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:05:11.309416 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.309245    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs podName:a282b450-2ac0-4aed-98ac-7be36ff01407 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:11.809223397 +0000 UTC m=+35.126680429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs") pod "router-default-6b946bb9b8-kxhmt" (UID: "a282b450-2ac0-4aed-98ac-7be36ff01407") : secret "router-metrics-certs-default" not found
Sep 11 13:05:11.309416 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.309251    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:05:11.309416 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.309315    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls podName:32e21675-4221-4ee4-a70f-5954bb73da27 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:11.809299887 +0000 UTC m=+35.126756927 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-v5c8r" (UID: "32e21675-4221-4ee4-a70f-5954bb73da27") : secret "samples-operator-tls" not found
Sep 11 13:05:11.309683 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.309165    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v5c8r\" (UID: \"32e21675-4221-4ee4-a70f-5954bb73da27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"
Sep 11 13:05:11.309683 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.309661    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dj5q\" (UniqueName: \"kubernetes.io/projected/54af5720-e75b-4e36-94cc-4a72fb166eb4-kube-api-access-7dj5q\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:11.309792 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.309690    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54af5720-e75b-4e36-94cc-4a72fb166eb4-config-volume\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:11.309792 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.309694    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-stats-auth\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.309792 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.309766    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a975d99e-85e6-4e33-b286-149c224c5d6e-config\") pod \"console-operator-696884fb9c-n5c4g\" (UID: \"a975d99e-85e6-4e33-b286-149c224c5d6e\") " pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.309930 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.309795    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.309930 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.309826    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbe4efbd-bb09-403a-a183-f0efd34611ee-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b65r7\" (UID: \"cbe4efbd-bb09-403a-a183-f0efd34611ee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"
Sep 11 13:05:11.309930 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.309875    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-default-certificate\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.309930 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.309924    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cbe4efbd-bb09-403a-a183-f0efd34611ee-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b65r7\" (UID: \"cbe4efbd-bb09-403a-a183-f0efd34611ee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"
Sep 11 13:05:11.310121 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.309967    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a975d99e-85e6-4e33-b286-149c224c5d6e-trusted-ca\") pod \"console-operator-696884fb9c-n5c4g\" (UID: \"a975d99e-85e6-4e33-b286-149c224c5d6e\") " pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.310121 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.310020    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:11.310121 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.310060    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls podName:54af5720-e75b-4e36-94cc-4a72fb166eb4 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:11.810046046 +0000 UTC m=+35.127503091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls") pod "dns-default-4hb2m" (UID: "54af5720-e75b-4e36-94cc-4a72fb166eb4") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:11.310309 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.310287    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle podName:a282b450-2ac0-4aed-98ac-7be36ff01407 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:11.810272709 +0000 UTC m=+35.127729749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle") pod "router-default-6b946bb9b8-kxhmt" (UID: "a282b450-2ac0-4aed-98ac-7be36ff01407") : configmap references non-existent config key: service-ca.crt
Sep 11 13:05:11.310963 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.310884    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a975d99e-85e6-4e33-b286-149c224c5d6e-config\") pod \"console-operator-696884fb9c-n5c4g\" (UID: \"a975d99e-85e6-4e33-b286-149c224c5d6e\") " pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.310963 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.310895    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54af5720-e75b-4e36-94cc-4a72fb166eb4-tmp-dir\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:11.311711 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.311685    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10e1239e-56fb-489b-9079-7b3c2e3619f9-cert\") pod \"ingress-canary-l7wdt\" (UID: \"10e1239e-56fb-489b-9079-7b3c2e3619f9\") " pod="openshift-ingress-canary/ingress-canary-l7wdt"
Sep 11 13:05:11.312384 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.312351    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-stats-auth\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.312577 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.312559    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-default-certificate\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.312857 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.312838    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a975d99e-85e6-4e33-b286-149c224c5d6e-serving-cert\") pod \"console-operator-696884fb9c-n5c4g\" (UID: \"a975d99e-85e6-4e33-b286-149c224c5d6e\") " pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.313115 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.313097    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cbe4efbd-bb09-403a-a183-f0efd34611ee-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b65r7\" (UID: \"cbe4efbd-bb09-403a-a183-f0efd34611ee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"
Sep 11 13:05:11.321436 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.321415    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkskg\" (UniqueName: \"kubernetes.io/projected/cbe4efbd-bb09-403a-a183-f0efd34611ee-kube-api-access-qkskg\") pod \"kube-storage-version-migrator-operator-6c8f84f897-b65r7\" (UID: \"cbe4efbd-bb09-403a-a183-f0efd34611ee\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"
Sep 11 13:05:11.321667 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.321646    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9q6b\" (UniqueName: \"kubernetes.io/projected/a975d99e-85e6-4e33-b286-149c224c5d6e-kube-api-access-p9q6b\") pod \"console-operator-696884fb9c-n5c4g\" (UID: \"a975d99e-85e6-4e33-b286-149c224c5d6e\") " pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.324432 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.324409    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lww6p\" (UniqueName: \"kubernetes.io/projected/32e21675-4221-4ee4-a70f-5954bb73da27-kube-api-access-lww6p\") pod \"cluster-samples-operator-86499c89c4-v5c8r\" (UID: \"32e21675-4221-4ee4-a70f-5954bb73da27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"
Sep 11 13:05:11.324613 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.324591    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqr86\" (UniqueName: \"kubernetes.io/projected/a282b450-2ac0-4aed-98ac-7be36ff01407-kube-api-access-wqr86\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.324758 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.324742    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9htw5\" (UniqueName: \"kubernetes.io/projected/10e1239e-56fb-489b-9079-7b3c2e3619f9-kube-api-access-9htw5\") pod \"ingress-canary-l7wdt\" (UID: \"10e1239e-56fb-489b-9079-7b3c2e3619f9\") " pod="openshift-ingress-canary/ingress-canary-l7wdt"
Sep 11 13:05:11.334597 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.334566    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dj5q\" (UniqueName: \"kubernetes.io/projected/54af5720-e75b-4e36-94cc-4a72fb166eb4-kube-api-access-7dj5q\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:11.365258 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.365232    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jfl55"
Sep 11 13:05:11.381257 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.381239    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-p49pp"
Sep 11 13:05:11.412325 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.412295    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:11.419102 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.419076    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"
Sep 11 13:05:11.426912 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.426892    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l7wdt"
Sep 11 13:05:11.712237 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.712205    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:11.712444 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.712314    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:11.712444 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.712396    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:05:11.712444 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.712414    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6548b9b645-mln9l: secret "image-registry-tls" not found
Sep 11 13:05:11.712444 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.712437    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:05:11.712621 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.712471    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls podName:458a0bb9-feca-4918-a2c4-799b4659f336 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:12.712450901 +0000 UTC m=+36.029907932 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls") pod "image-registry-6548b9b645-mln9l" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336") : secret "image-registry-tls" not found
Sep 11 13:05:11.712621 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.712491    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls podName:6e4b7f10-74fd-40fd-ba55-61080037f297 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:12.712481385 +0000 UTC m=+36.029938403 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4rqbt" (UID: "6e4b7f10-74fd-40fd-ba55-61080037f297") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:05:11.813496 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.813460    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:11.814181 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.813536    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v5c8r\" (UID: \"32e21675-4221-4ee4-a70f-5954bb73da27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"
Sep 11 13:05:11.814181 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.813583    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.814181 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.813634    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:11.814181 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:11.813652    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:11.814181 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.813663    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:05:11.814181 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.813704    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls podName:54af5720-e75b-4e36-94cc-4a72fb166eb4 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:12.813685215 +0000 UTC m=+36.131142257 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls") pod "dns-default-4hb2m" (UID: "54af5720-e75b-4e36-94cc-4a72fb166eb4") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:11.814181 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.813720    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls podName:32e21675-4221-4ee4-a70f-5954bb73da27 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:12.81371386 +0000 UTC m=+36.131170877 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-v5c8r" (UID: "32e21675-4221-4ee4-a70f-5954bb73da27") : secret "samples-operator-tls" not found
Sep 11 13:05:11.814181 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.813754    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:05:11.814181 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.813776    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle podName:a282b450-2ac0-4aed-98ac-7be36ff01407 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:12.813757714 +0000 UTC m=+36.131214748 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle") pod "router-default-6b946bb9b8-kxhmt" (UID: "a282b450-2ac0-4aed-98ac-7be36ff01407") : configmap references non-existent config key: service-ca.crt
Sep 11 13:05:11.814181 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:11.813796    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs podName:a282b450-2ac0-4aed-98ac-7be36ff01407 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:12.813787274 +0000 UTC m=+36.131244291 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs") pod "router-default-6b946bb9b8-kxhmt" (UID: "a282b450-2ac0-4aed-98ac-7be36ff01407") : secret "router-metrics-certs-default" not found
Sep 11 13:05:12.647091 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:12.646852    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7"]
Sep 11 13:05:12.649825 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:12.649802    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jfl55"]
Sep 11 13:05:12.655935 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:12.655915    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-p49pp"]
Sep 11 13:05:12.656645 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:12.656628    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l7wdt"]
Sep 11 13:05:12.661462 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:12.661445    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-n5c4g"]
Sep 11 13:05:12.666572 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:05:12.666539    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa53386d_9f0f_4d4e_868f_520ca3f17cde.slice/crio-4aa56c2f3197c0ecd28bf82a24de082e5afe3b7ed6987a3514546ef4dcce4c78 WatchSource:0}: Error finding container 4aa56c2f3197c0ecd28bf82a24de082e5afe3b7ed6987a3514546ef4dcce4c78: Status 404 returned error can't find the container with id 4aa56c2f3197c0ecd28bf82a24de082e5afe3b7ed6987a3514546ef4dcce4c78
Sep 11 13:05:12.667480 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:05:12.667435    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda975d99e_85e6_4e33_b286_149c224c5d6e.slice/crio-4f4f6239dd709a276367a076303adc419e736b947dfcc54b5a7c172bb292ccae WatchSource:0}: Error finding container 4f4f6239dd709a276367a076303adc419e736b947dfcc54b5a7c172bb292ccae: Status 404 returned error can't find the container with id 4f4f6239dd709a276367a076303adc419e736b947dfcc54b5a7c172bb292ccae
Sep 11 13:05:12.668711 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:05:12.668683    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10e1239e_56fb_489b_9079_7b3c2e3619f9.slice/crio-b3ce03a15e277aa04a62f224be40a5345b24e2584f4c4ad286d33aa6cadf9632 WatchSource:0}: Error finding container b3ce03a15e277aa04a62f224be40a5345b24e2584f4c4ad286d33aa6cadf9632: Status 404 returned error can't find the container with id b3ce03a15e277aa04a62f224be40a5345b24e2584f4c4ad286d33aa6cadf9632
Sep 11 13:05:12.669826 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:05:12.669806    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cf01a79_1e79_45b8_8cd5_1ee9e4e8117e.slice/crio-110af380c7fbbf8566aa16f9934316dc4d41828b3530fce84dee49e2020ae130 WatchSource:0}: Error finding container 110af380c7fbbf8566aa16f9934316dc4d41828b3530fce84dee49e2020ae130: Status 404 returned error can't find the container with id 110af380c7fbbf8566aa16f9934316dc4d41828b3530fce84dee49e2020ae130
Sep 11 13:05:12.670182 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:05:12.670151    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbe4efbd_bb09_403a_a183_f0efd34611ee.slice/crio-95492b0465b495e0ff2ef08b31cb7949d006dda991dc0ca4c42f7bf9510568b0 WatchSource:0}: Error finding container 95492b0465b495e0ff2ef08b31cb7949d006dda991dc0ca4c42f7bf9510568b0: Status 404 returned error can't find the container with id 95492b0465b495e0ff2ef08b31cb7949d006dda991dc0ca4c42f7bf9510568b0
Sep 11 13:05:12.723134 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:12.723106    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:12.723256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:12.723179    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:12.723256 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:12.723245    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:05:12.723332 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:12.723298    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:05:12.723332 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:12.723311    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6548b9b645-mln9l: secret "image-registry-tls" not found
Sep 11 13:05:12.723420 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:12.723302    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls podName:6e4b7f10-74fd-40fd-ba55-61080037f297 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:14.723288389 +0000 UTC m=+38.040745406 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4rqbt" (UID: "6e4b7f10-74fd-40fd-ba55-61080037f297") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:05:12.723420 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:12.723357    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls podName:458a0bb9-feca-4918-a2c4-799b4659f336 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:14.723343555 +0000 UTC m=+38.040800574 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls") pod "image-registry-6548b9b645-mln9l" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336") : secret "image-registry-tls" not found
Sep 11 13:05:12.823782 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:12.823755    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:12.824283 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:12.823804    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:12.824283 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:12.823921    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v5c8r\" (UID: \"32e21675-4221-4ee4-a70f-5954bb73da27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"
Sep 11 13:05:12.824283 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:12.823936    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:05:12.824283 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:12.823953    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:12.824283 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:12.824009    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs podName:a282b450-2ac0-4aed-98ac-7be36ff01407 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:14.823988404 +0000 UTC m=+38.141445425 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs") pod "router-default-6b946bb9b8-kxhmt" (UID: "a282b450-2ac0-4aed-98ac-7be36ff01407") : secret "router-metrics-certs-default" not found
Sep 11 13:05:12.824283 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:12.824044    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:12.824283 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:12.824051    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:05:12.824283 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:12.824079    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle podName:a282b450-2ac0-4aed-98ac-7be36ff01407 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:14.824063191 +0000 UTC m=+38.141520228 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle") pod "router-default-6b946bb9b8-kxhmt" (UID: "a282b450-2ac0-4aed-98ac-7be36ff01407") : configmap references non-existent config key: service-ca.crt
Sep 11 13:05:12.824283 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:12.824102    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls podName:32e21675-4221-4ee4-a70f-5954bb73da27 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:14.824088195 +0000 UTC m=+38.141545214 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-v5c8r" (UID: "32e21675-4221-4ee4-a70f-5954bb73da27") : secret "samples-operator-tls" not found
Sep 11 13:05:12.824283 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:12.824118    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls podName:54af5720-e75b-4e36-94cc-4a72fb166eb4 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:14.824111466 +0000 UTC m=+38.141568485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls") pod "dns-default-4hb2m" (UID: "54af5720-e75b-4e36-94cc-4a72fb166eb4") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:13.342620 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:13.342576    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jfl55" event={"ID":"aa53386d-9f0f-4d4e-868f-520ca3f17cde","Type":"ContainerStarted","Data":"4aa56c2f3197c0ecd28bf82a24de082e5afe3b7ed6987a3514546ef4dcce4c78"}
Sep 11 13:05:13.347147 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:13.347112    2554 generic.go:356] "Generic (PLEG): container finished" podID="b0312884-f43d-4476-8501-83abe961b238" containerID="2a352e27c8c38c05492a109d9b5da2e63923a5d8571643498a7d15e7f76adbdc" exitCode=0
Sep 11 13:05:13.347280 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:13.347200    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gt6zk" event={"ID":"b0312884-f43d-4476-8501-83abe961b238","Type":"ContainerDied","Data":"2a352e27c8c38c05492a109d9b5da2e63923a5d8571643498a7d15e7f76adbdc"}
Sep 11 13:05:13.350857 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:13.350781    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g" event={"ID":"a975d99e-85e6-4e33-b286-149c224c5d6e","Type":"ContainerStarted","Data":"4f4f6239dd709a276367a076303adc419e736b947dfcc54b5a7c172bb292ccae"}
Sep 11 13:05:13.362880 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:13.362797    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l7wdt" event={"ID":"10e1239e-56fb-489b-9079-7b3c2e3619f9","Type":"ContainerStarted","Data":"b3ce03a15e277aa04a62f224be40a5345b24e2584f4c4ad286d33aa6cadf9632"}
Sep 11 13:05:13.364725 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:13.364691    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-p49pp" event={"ID":"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e","Type":"ContainerStarted","Data":"110af380c7fbbf8566aa16f9934316dc4d41828b3530fce84dee49e2020ae130"}
Sep 11 13:05:13.366145 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:13.366113    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7" event={"ID":"cbe4efbd-bb09-403a-a183-f0efd34611ee","Type":"ContainerStarted","Data":"95492b0465b495e0ff2ef08b31cb7949d006dda991dc0ca4c42f7bf9510568b0"}
Sep 11 13:05:14.373973 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:14.373715    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jfl55" event={"ID":"aa53386d-9f0f-4d4e-868f-520ca3f17cde","Type":"ContainerStarted","Data":"f20dcccefc0522bdcc690a4ae270e6a5a0317556ada7236fa234f3112dd00021"}
Sep 11 13:05:14.381008 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:14.380976    2554 generic.go:356] "Generic (PLEG): container finished" podID="b0312884-f43d-4476-8501-83abe961b238" containerID="ddc6acf329cede5045910ae81beded63ea0433ad0a5d91a51e1e5f31f3711b44" exitCode=0
Sep 11 13:05:14.381143 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:14.381035    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gt6zk" event={"ID":"b0312884-f43d-4476-8501-83abe961b238","Type":"ContainerDied","Data":"ddc6acf329cede5045910ae81beded63ea0433ad0a5d91a51e1e5f31f3711b44"}
Sep 11 13:05:14.396862 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:14.395390    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jfl55" podStartSLOduration=5.1049759 podStartE2EDuration="6.395313387s" totalImagesPullingTime="1.290337487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:14.394619271 +0000 UTC m=+37.712076312" watchObservedRunningTime="2026-09-11 13:05:14.395313387 +0000 UTC m=+37.712770431"
Sep 11 13:05:14.563251 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:14.563214    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:14.692033 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:14.691946    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:14.744649 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:14.744604    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:14.744842 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:14.744732    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:14.744842 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:14.744826    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:05:14.744959 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:14.744888    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls podName:6e4b7f10-74fd-40fd-ba55-61080037f297 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:18.744873659 +0000 UTC m=+42.062330690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4rqbt" (UID: "6e4b7f10-74fd-40fd-ba55-61080037f297") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:05:14.744959 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:14.744832    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:05:14.744959 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:14.744938    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6548b9b645-mln9l: secret "image-registry-tls" not found
Sep 11 13:05:14.745120 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:14.744979    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls podName:458a0bb9-feca-4918-a2c4-799b4659f336 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:18.744967287 +0000 UTC m=+42.062424312 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls") pod "image-registry-6548b9b645-mln9l" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336") : secret "image-registry-tls" not found
Sep 11 13:05:14.845318 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:14.845281    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v5c8r\" (UID: \"32e21675-4221-4ee4-a70f-5954bb73da27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"
Sep 11 13:05:14.845504 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:14.845350    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:14.845504 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:14.845460    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:14.845504 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:14.845472    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:05:14.845504 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:14.845489    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle podName:a282b450-2ac0-4aed-98ac-7be36ff01407 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:18.845468583 +0000 UTC m=+42.162925616 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle") pod "router-default-6b946bb9b8-kxhmt" (UID: "a282b450-2ac0-4aed-98ac-7be36ff01407") : configmap references non-existent config key: service-ca.crt
Sep 11 13:05:14.845721 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:14.845520    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls podName:32e21675-4221-4ee4-a70f-5954bb73da27 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:18.845508434 +0000 UTC m=+42.162965454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-v5c8r" (UID: "32e21675-4221-4ee4-a70f-5954bb73da27") : secret "samples-operator-tls" not found
Sep 11 13:05:14.845721 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:14.845547    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:05:14.845721 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:14.845577    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:14.845721 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:14.845601    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs podName:a282b450-2ac0-4aed-98ac-7be36ff01407 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:18.845586217 +0000 UTC m=+42.163043240 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs") pod "router-default-6b946bb9b8-kxhmt" (UID: "a282b450-2ac0-4aed-98ac-7be36ff01407") : secret "router-metrics-certs-default" not found
Sep 11 13:05:14.845721 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:14.845636    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:14.845721 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:14.845676    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls podName:54af5720-e75b-4e36-94cc-4a72fb166eb4 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:18.845665661 +0000 UTC m=+42.163122701 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls") pod "dns-default-4hb2m" (UID: "54af5720-e75b-4e36-94cc-4a72fb166eb4") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:15.044488 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:15.044447    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:15.198834 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:15.198797    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:15.389098 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:15.389061    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gt6zk" event={"ID":"b0312884-f43d-4476-8501-83abe961b238","Type":"ContainerStarted","Data":"52a7109cd31c701603fadc1ad770588252d5f645b98e4496dd7b9c1bd8619fcc"}
Sep 11 13:05:15.431985 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:15.431873    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gt6zk" podStartSLOduration=9.382440923 podStartE2EDuration="38.431849111s" totalImagesPullingTime="29.049408188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:15.430457775 +0000 UTC m=+38.747914816" watchObservedRunningTime="2026-09-11 13:05:15.431849111 +0000 UTC m=+38.749306163"
Sep 11 13:05:15.655668 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:15.655587    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:15.819133 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:15.819098    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:16.070513 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:16.070483    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:16.482996 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:16.482863    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:16.613546 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:16.613511    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:17.414568 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:17.414534    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:17.603899 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:17.603847    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:17.739942 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:17.739864    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:17.953734 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:17.953706    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:18.052789 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.052762    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:18.153160 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.153094    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:18.313264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.313182    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:18.398382 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.398334    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7" event={"ID":"cbe4efbd-bb09-403a-a183-f0efd34611ee","Type":"ContainerStarted","Data":"f7092f796ba765d20ed383e813e161f2781819e8baa81f2f5df5237d3c3a630a"}
Sep 11 13:05:18.400255 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.400230    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/0.log"
Sep 11 13:05:18.400405 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.400276    2554 generic.go:356] "Generic (PLEG): container finished" podID="a975d99e-85e6-4e33-b286-149c224c5d6e" containerID="ada0addda07cd34cbab15344dfb0ebb1d7865904aa998f7f04e23a39d857bb84" exitCode=255
Sep 11 13:05:18.400405 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.400355    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g" event={"ID":"a975d99e-85e6-4e33-b286-149c224c5d6e","Type":"ContainerDied","Data":"ada0addda07cd34cbab15344dfb0ebb1d7865904aa998f7f04e23a39d857bb84"}
Sep 11 13:05:18.400563 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.400546    2554 scope.go:117] "RemoveContainer" containerID="ada0addda07cd34cbab15344dfb0ebb1d7865904aa998f7f04e23a39d857bb84"
Sep 11 13:05:18.402112 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.402087    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l7wdt" event={"ID":"10e1239e-56fb-489b-9079-7b3c2e3619f9","Type":"ContainerStarted","Data":"7c5686c8b7dd59dd8188d3a65a7e36997f9db75de0a88084c902e34080157f19"}
Sep 11 13:05:18.403710 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.403691    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-p49pp" event={"ID":"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e","Type":"ContainerStarted","Data":"5fd092ac842beb12aaba4a6cec2ff80f2255c3775a457e90caf8b9eb6ea3d767"}
Sep 11 13:05:18.414933 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.414910    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:18.454048 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.453999    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7" podStartSLOduration=3.231284688 podStartE2EDuration="8.453982228s" totalImagesPullingTime="5.22269754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:18.430396301 +0000 UTC m=+41.747853346" watchObservedRunningTime="2026-09-11 13:05:18.453982228 +0000 UTC m=+41.771439269"
Sep 11 13:05:18.454426 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.454354    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-p49pp" podStartSLOduration=6.231142372 podStartE2EDuration="11.454346062s" totalImagesPullingTime="5.22320369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:18.453047453 +0000 UTC m=+41.770504493" watchObservedRunningTime="2026-09-11 13:05:18.454346062 +0000 UTC m=+41.771803101"
Sep 11 13:05:18.477052 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.477000    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-l7wdt" podStartSLOduration=2.253195939 podStartE2EDuration="7.4769798s" totalImagesPullingTime="5.223783861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:18.475159358 +0000 UTC m=+41.792616399" watchObservedRunningTime="2026-09-11 13:05:18.4769798 +0000 UTC m=+41.794436841"
Sep 11 13:05:18.782589 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.782550    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:18.782982 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.782623    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:18.782982 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:18.782725    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:05:18.782982 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:18.782730    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:05:18.782982 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:18.782736    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6548b9b645-mln9l: secret "image-registry-tls" not found
Sep 11 13:05:18.782982 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:18.782862    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls podName:6e4b7f10-74fd-40fd-ba55-61080037f297 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:26.782839082 +0000 UTC m=+50.100296113 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4rqbt" (UID: "6e4b7f10-74fd-40fd-ba55-61080037f297") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:05:18.782982 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:18.782883    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls podName:458a0bb9-feca-4918-a2c4-799b4659f336 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:26.782872555 +0000 UTC m=+50.100329580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls") pod "image-registry-6548b9b645-mln9l" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336") : secret "image-registry-tls" not found
Sep 11 13:05:18.883886 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.883811    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:18.883886 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.883870    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:18.884093 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.883931    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v5c8r\" (UID: \"32e21675-4221-4ee4-a70f-5954bb73da27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"
Sep 11 13:05:18.884093 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:18.883958    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:05:18.884093 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:18.883974    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:18.884093 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:18.884025    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs podName:a282b450-2ac0-4aed-98ac-7be36ff01407 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:26.88400663 +0000 UTC m=+50.201463662 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs") pod "router-default-6b946bb9b8-kxhmt" (UID: "a282b450-2ac0-4aed-98ac-7be36ff01407") : secret "router-metrics-certs-default" not found
Sep 11 13:05:18.884093 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:18.884041    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:18.884093 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:18.884068    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle podName:a282b450-2ac0-4aed-98ac-7be36ff01407 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:26.884057929 +0000 UTC m=+50.201514964 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle") pod "router-default-6b946bb9b8-kxhmt" (UID: "a282b450-2ac0-4aed-98ac-7be36ff01407") : configmap references non-existent config key: service-ca.crt
Sep 11 13:05:18.884324 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:18.884100    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:05:18.884324 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:18.884118    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls podName:54af5720-e75b-4e36-94cc-4a72fb166eb4 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:26.884085914 +0000 UTC m=+50.201542945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls") pod "dns-default-4hb2m" (UID: "54af5720-e75b-4e36-94cc-4a72fb166eb4") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:18.884324 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:18.884167    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls podName:32e21675-4221-4ee4-a70f-5954bb73da27 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:26.884152527 +0000 UTC m=+50.201609560 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-v5c8r" (UID: "32e21675-4221-4ee4-a70f-5954bb73da27") : secret "samples-operator-tls" not found
Sep 11 13:05:19.352741 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.352694    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mp69k"]
Sep 11 13:05:19.357184 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.357156    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:19.373259 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.373233    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:19.374346 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.374323    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mp69k"]
Sep 11 13:05:19.376057 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.376039    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jddd5\""
Sep 11 13:05:19.376319 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.376294    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 13:05:19.376319 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.376311    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 13:05:19.408573 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.408549    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:05:19.409077 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.409056    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/0.log"
Sep 11 13:05:19.409172 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.409096    2554 generic.go:356] "Generic (PLEG): container finished" podID="a975d99e-85e6-4e33-b286-149c224c5d6e" containerID="8d902cedec9e15f615b27d1f7dd746d593493075c80734890373824f3c34572a" exitCode=255
Sep 11 13:05:19.409234 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.409213    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g" event={"ID":"a975d99e-85e6-4e33-b286-149c224c5d6e","Type":"ContainerDied","Data":"8d902cedec9e15f615b27d1f7dd746d593493075c80734890373824f3c34572a"}
Sep 11 13:05:19.409301 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.409282    2554 scope.go:117] "RemoveContainer" containerID="ada0addda07cd34cbab15344dfb0ebb1d7865904aa998f7f04e23a39d857bb84"
Sep 11 13:05:19.409569 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.409538    2554 scope.go:117] "RemoveContainer" containerID="8d902cedec9e15f615b27d1f7dd746d593493075c80734890373824f3c34572a"
Sep 11 13:05:19.409773 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:19.409747    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-n5c4g_openshift-console-operator(a975d99e-85e6-4e33-b286-149c224c5d6e)\"" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g" podUID="a975d99e-85e6-4e33-b286-149c224c5d6e"
Sep 11 13:05:19.424166 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.424123    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:19.488943 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.488896    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:19.488943 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.488944    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/663c3e69-ad92-4794-b0c4-9b8c319b176c-crio-socket\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:19.489172 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.489060    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:19.489172 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.489107    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svtzb\" (UniqueName: \"kubernetes.io/projected/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-api-access-svtzb\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:19.489172 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.489143    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/663c3e69-ad92-4794-b0c4-9b8c319b176c-data-volume\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:19.590214 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.590168    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:19.590460 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.590269    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svtzb\" (UniqueName: \"kubernetes.io/projected/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-api-access-svtzb\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:19.590460 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:19.590303    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:19.590460 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:19.590408    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm podName:663c3e69-ad92-4794-b0c4-9b8c319b176c nodeName:}" failed. No retries permitted until 2026-09-11 13:05:20.090384783 +0000 UTC m=+43.407841815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mp69k" (UID: "663c3e69-ad92-4794-b0c4-9b8c319b176c") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:19.590628 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.590460    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/663c3e69-ad92-4794-b0c4-9b8c319b176c-data-volume\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:19.590895 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.590867    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:19.591045 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.590931    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/663c3e69-ad92-4794-b0c4-9b8c319b176c-crio-socket\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:19.591045 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:19.590958    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:19.591045 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:19.591002    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls podName:663c3e69-ad92-4794-b0c4-9b8c319b176c nodeName:}" failed. No retries permitted until 2026-09-11 13:05:20.090989 +0000 UTC m=+43.408446024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mp69k" (UID: "663c3e69-ad92-4794-b0c4-9b8c319b176c") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:19.591045 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.591005    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/663c3e69-ad92-4794-b0c4-9b8c319b176c-data-volume\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:19.591225 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.591101    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/663c3e69-ad92-4794-b0c4-9b8c319b176c-crio-socket\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:19.600960 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:19.600928    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svtzb\" (UniqueName: \"kubernetes.io/projected/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-api-access-svtzb\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:20.017493 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:20.017454    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:20.096219 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:20.096187    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:20.096377 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:20.096345    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:20.096428 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:20.096381    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:20.096511 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:20.096493    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:20.096551 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:20.096519    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls podName:663c3e69-ad92-4794-b0c4-9b8c319b176c nodeName:}" failed. No retries permitted until 2026-09-11 13:05:21.096504795 +0000 UTC m=+44.413961814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mp69k" (UID: "663c3e69-ad92-4794-b0c4-9b8c319b176c") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:20.096591 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:20.096552    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm podName:663c3e69-ad92-4794-b0c4-9b8c319b176c nodeName:}" failed. No retries permitted until 2026-09-11 13:05:21.096539001 +0000 UTC m=+44.413996033 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mp69k" (UID: "663c3e69-ad92-4794-b0c4-9b8c319b176c") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:20.122262 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:20.122214    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:20.264651 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:20.264610    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:20.413246 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:20.413219    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:05:20.413627 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:20.413609    2554 scope.go:117] "RemoveContainer" containerID="8d902cedec9e15f615b27d1f7dd746d593493075c80734890373824f3c34572a"
Sep 11 13:05:20.413812 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:20.413792    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-n5c4g_openshift-console-operator(a975d99e-85e6-4e33-b286-149c224c5d6e)\"" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g" podUID="a975d99e-85e6-4e33-b286-149c224c5d6e"
Sep 11 13:05:20.904073 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:20.904042    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-twqp9_3b49a550-e226-4de1-8351-a9fd45b4f76f/dns-node-resolver/0.log"
Sep 11 13:05:21.104635 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:21.104595    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:21.105118 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:21.104693    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:21.105118 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:21.104779    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:21.105118 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:21.104809    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:21.105118 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:21.104867    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls podName:663c3e69-ad92-4794-b0c4-9b8c319b176c nodeName:}" failed. No retries permitted until 2026-09-11 13:05:23.10484669 +0000 UTC m=+46.422303711 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mp69k" (UID: "663c3e69-ad92-4794-b0c4-9b8c319b176c") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:21.105118 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:21.104888    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm podName:663c3e69-ad92-4794-b0c4-9b8c319b176c nodeName:}" failed. No retries permitted until 2026-09-11 13:05:23.104877671 +0000 UTC m=+46.422334690 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mp69k" (UID: "663c3e69-ad92-4794-b0c4-9b8c319b176c") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:21.413337 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:21.413294    2554 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:21.413551 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:21.413353    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:21.415536 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:21.415516    2554 scope.go:117] "RemoveContainer" containerID="8d902cedec9e15f615b27d1f7dd746d593493075c80734890373824f3c34572a"
Sep 11 13:05:21.415681 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:21.415664    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-n5c4g_openshift-console-operator(a975d99e-85e6-4e33-b286-149c224c5d6e)\"" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g" podUID="a975d99e-85e6-4e33-b286-149c224c5d6e"
Sep 11 13:05:21.449141 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:21.449106    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:22.099302 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:22.099275    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x89jp_1572be9a-6852-40f5-951c-b0a82cd9961f/node-ca/0.log"
Sep 11 13:05:22.499821 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:22.499747    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l7wdt_10e1239e-56fb-489b-9079-7b3c2e3619f9/serve-healthcheck-canary/0.log"
Sep 11 13:05:23.122089 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:23.122049    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:23.122287 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:23.122164    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:23.122287 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:23.122208    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:23.122287 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:23.122273    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:23.122442 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:23.122277    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls podName:663c3e69-ad92-4794-b0c4-9b8c319b176c nodeName:}" failed. No retries permitted until 2026-09-11 13:05:27.122262172 +0000 UTC m=+50.439719191 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mp69k" (UID: "663c3e69-ad92-4794-b0c4-9b8c319b176c") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:23.122442 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:23.122335    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm podName:663c3e69-ad92-4794-b0c4-9b8c319b176c nodeName:}" failed. No retries permitted until 2026-09-11 13:05:27.12232058 +0000 UTC m=+50.439777598 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mp69k" (UID: "663c3e69-ad92-4794-b0c4-9b8c319b176c") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:23.919620 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:23.919586    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:25.434489 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:25.434443    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:25.530972 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:25.530909    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:25.667594 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:25.667556    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:25.808527 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:25.808453    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:25.912836 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:25.912804    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:26.139380 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:26.139335    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:26.853793 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:26.853758    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:26.854169 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:26.853906    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:05:26.854169 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:26.853938    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:26.854169 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:26.853977    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls podName:6e4b7f10-74fd-40fd-ba55-61080037f297 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:42.853957397 +0000 UTC m=+66.171414440 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4rqbt" (UID: "6e4b7f10-74fd-40fd-ba55-61080037f297") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:05:26.854169 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:26.854021    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:05:26.854169 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:26.854033    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6548b9b645-mln9l: secret "image-registry-tls" not found
Sep 11 13:05:26.854169 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:26.854077    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls podName:458a0bb9-feca-4918-a2c4-799b4659f336 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:42.854064199 +0000 UTC m=+66.171521225 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls") pod "image-registry-6548b9b645-mln9l" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336") : secret "image-registry-tls" not found
Sep 11 13:05:26.916628 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:26.916594    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:26.954791 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:26.954758    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:26.954955 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:26.954823    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:26.954955 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:26.954865    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v5c8r\" (UID: \"32e21675-4221-4ee4-a70f-5954bb73da27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"
Sep 11 13:05:26.954955 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:26.954889    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:05:26.955126 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:26.954959    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs podName:a282b450-2ac0-4aed-98ac-7be36ff01407 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:42.954938852 +0000 UTC m=+66.272395880 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs") pod "router-default-6b946bb9b8-kxhmt" (UID: "a282b450-2ac0-4aed-98ac-7be36ff01407") : secret "router-metrics-certs-default" not found
Sep 11 13:05:26.955126 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:26.954981    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:26.955126 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:26.954892    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:26.955126 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:26.954983    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle podName:a282b450-2ac0-4aed-98ac-7be36ff01407 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:42.954968588 +0000 UTC m=+66.272425622 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle") pod "router-default-6b946bb9b8-kxhmt" (UID: "a282b450-2ac0-4aed-98ac-7be36ff01407") : configmap references non-existent config key: service-ca.crt
Sep 11 13:05:26.955126 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:26.955063    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls podName:54af5720-e75b-4e36-94cc-4a72fb166eb4 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:42.955049121 +0000 UTC m=+66.272506145 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls") pod "dns-default-4hb2m" (UID: "54af5720-e75b-4e36-94cc-4a72fb166eb4") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:26.957397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:26.957354    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/32e21675-4221-4ee4-a70f-5954bb73da27-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v5c8r\" (UID: \"32e21675-4221-4ee4-a70f-5954bb73da27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"
Sep 11 13:05:26.996656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:26.996631    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"
Sep 11 13:05:27.155975 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:27.155895    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:27.155975 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:27.155970    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:27.156174 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:27.156029    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:27.156174 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:27.156048    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:27.156174 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:27.156086    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls podName:663c3e69-ad92-4794-b0c4-9b8c319b176c nodeName:}" failed. No retries permitted until 2026-09-11 13:05:35.156071671 +0000 UTC m=+58.473528690 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mp69k" (UID: "663c3e69-ad92-4794-b0c4-9b8c319b176c") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:27.156174 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:27.156099    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm podName:663c3e69-ad92-4794-b0c4-9b8c319b176c nodeName:}" failed. No retries permitted until 2026-09-11 13:05:35.156093744 +0000 UTC m=+58.473550761 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mp69k" (UID: "663c3e69-ad92-4794-b0c4-9b8c319b176c") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:27.315832 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:27.315799    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r"]
Sep 11 13:05:27.341866 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:27.341842    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:27.430411 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:27.430338    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r" event={"ID":"32e21675-4221-4ee4-a70f-5954bb73da27","Type":"ContainerStarted","Data":"b43f256b8bd475fa68c3c840b161810e23909f074e696bd385a845d67343ab12"}
Sep 11 13:05:27.711287 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:27.711213    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:27.893522 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:27.893489    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:28.141447 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:28.141412    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:28.510220 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:28.510140    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:28.845710 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:28.845681    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:29.107640 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:29.107560    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:29.207052 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:29.207022    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:29.306515 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:29.306476    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:29.489665 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:29.489567    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:29.690622 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:29.690590    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:29.792193 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:29.792165    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:30.440885 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:30.440846    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r" event={"ID":"32e21675-4221-4ee4-a70f-5954bb73da27","Type":"ContainerStarted","Data":"a80b3662bfd1c3e060b0243480f0f580567be7217d7904f4c186395575bf691d"}
Sep 11 13:05:30.441266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:30.440926    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r" event={"ID":"32e21675-4221-4ee4-a70f-5954bb73da27","Type":"ContainerStarted","Data":"460d8780426e7ae656482819d7522a3448c80ca68c1b08e7a14cc1c80b649f4d"}
Sep 11 13:05:30.464184 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:30.464139    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v5c8r" podStartSLOduration=20.114450128 podStartE2EDuration="22.464124258s" totalImagesPullingTime="2.34967413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:30.463578988 +0000 UTC m=+53.781036028" watchObservedRunningTime="2026-09-11 13:05:30.464124258 +0000 UTC m=+53.781581298"
Sep 11 13:05:30.901509 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:30.901463    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:31.122450 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:31.122404    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:31.283053 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:31.282972    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:31.381751 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:31.381712    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:31.507462 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:31.507433    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:31.694269 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:31.694239    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:32.904755 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:32.904723    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:33.004131 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:33.004100    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:33.105412 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:33.105380    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:33.311761 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:33.311684    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:33.494810 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:33.494780    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:33.697775 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:33.697741    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:34.695077 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:34.695047    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:35.088282 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:35.088250    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:35.226964 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:35.226923    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:35.227158 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:35.227017    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:35.227158 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:35.227122    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:35.227279 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:05:35.227206    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls podName:663c3e69-ad92-4794-b0c4-9b8c319b176c nodeName:}" failed. No retries permitted until 2026-09-11 13:05:51.227181609 +0000 UTC m=+74.544638627 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mp69k" (UID: "663c3e69-ad92-4794-b0c4-9b8c319b176c") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:35.227589 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:35.227564    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/663c3e69-ad92-4794-b0c4-9b8c319b176c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:35.292498 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:35.292467    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:35.515623 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:35.515519    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:35.701824 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:35.701784    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:36.094217 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:36.094185    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:36.147483 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:36.147457    2554 scope.go:117] "RemoveContainer" containerID="8d902cedec9e15f615b27d1f7dd746d593493075c80734890373824f3c34572a"
Sep 11 13:05:36.461717 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:36.461637    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:05:36.461892 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:36.461720    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g" event={"ID":"a975d99e-85e6-4e33-b286-149c224c5d6e","Type":"ContainerStarted","Data":"197ddbf17e652b1245bb0606999b3df38e2c2829515544f6b37fc4139a56f783"}
Sep 11 13:05:36.462018 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:36.461994    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:36.481254 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:36.481199    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g" podStartSLOduration=23.251994649 podStartE2EDuration="28.481184557s" totalImagesPullingTime="5.229189908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:36.480482493 +0000 UTC m=+59.797939533" watchObservedRunningTime="2026-09-11 13:05:36.481184557 +0000 UTC m=+59.798641646"
Sep 11 13:05:36.798210 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:36.798135    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-n5c4g"
Sep 11 13:05:37.115099 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:37.115064    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:37.236682 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:37.236639    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:37.332465 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:37.332437    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-46vhh"
Sep 11 13:05:37.335387 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:37.335345    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:37.472727 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:37.472646    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:37.615820 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:37.615792    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:37.884071 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:37.884035    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:37.974586 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:37.974545    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:38.294534 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:38.294459    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:38.377440 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:38.377402    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:38.884747 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:38.884704    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:39.089886 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:39.089858    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:39.190194 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:39.190109    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:40.100869 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:40.100839    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:40.202483 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:40.202445    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:41.260556 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:41.260518    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:41.404216 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:41.404177    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:41.524793 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:41.524714    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:41.642015 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:41.641980    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:42.902816 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:42.902778    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:42.903264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:42.902823    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:05:42.903264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:42.903001    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:42.905276 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:42.905247    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls\") pod \"image-registry-6548b9b645-mln9l\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") " pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:42.905276 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:42.905263    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e4b7f10-74fd-40fd-ba55-61080037f297-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4rqbt\" (UID: \"6e4b7f10-74fd-40fd-ba55-61080037f297\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:42.905631 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:42.905617    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:05:42.915330 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:42.915310    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6942647-056e-4b32-a2b4-ea2e90f7be76-original-pull-secret\") pod \"global-pull-secret-syncer-p5hww\" (UID: \"c6942647-056e-4b32-a2b4-ea2e90f7be76\") " pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:05:42.972025 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:42.971986    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p5hww"
Sep 11 13:05:43.004032 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.004002    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:43.004173 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.004050    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:43.004173 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.004074    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ntnk\" (UniqueName: \"kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk\") pod \"network-check-target-qbm58\" (UID: \"439302b6-1aa6-4530-9e4f-3534c305b8e1\") " pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:05:43.004173 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.004093    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs\") pod \"network-metrics-daemon-9rsjp\" (UID: \"0498fed7-8022-4a6b-9e69-3a560c5f5e02\") " pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:05:43.004173 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.004119    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:43.004792 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.004770    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a282b450-2ac0-4aed-98ac-7be36ff01407-service-ca-bundle\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:43.007018 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.006985    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a282b450-2ac0-4aed-98ac-7be36ff01407-metrics-certs\") pod \"router-default-6b946bb9b8-kxhmt\" (UID: \"a282b450-2ac0-4aed-98ac-7be36ff01407\") " pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:43.007144 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.007124    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54af5720-e75b-4e36-94cc-4a72fb166eb4-metrics-tls\") pod \"dns-default-4hb2m\" (UID: \"54af5720-e75b-4e36-94cc-4a72fb166eb4\") " pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:43.007739 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.007719    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:05:43.008396 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.008377    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:05:43.016537 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.016497    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0498fed7-8022-4a6b-9e69-3a560c5f5e02-metrics-certs\") pod \"network-metrics-daemon-9rsjp\" (UID: \"0498fed7-8022-4a6b-9e69-3a560c5f5e02\") " pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:05:43.016699 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.016665    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:05:43.026843 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.026811    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ntnk\" (UniqueName: \"kubernetes.io/projected/439302b6-1aa6-4530-9e4f-3534c305b8e1-kube-api-access-8ntnk\") pod \"network-check-target-qbm58\" (UID: \"439302b6-1aa6-4530-9e4f-3534c305b8e1\") " pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:05:43.158823 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.158741    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zxvfq\""
Sep 11 13:05:43.166904 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.166879    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"
Sep 11 13:05:43.175517 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.175492    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-crft9\""
Sep 11 13:05:43.183417 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.183388    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:43.207913 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.207892    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tffbk\""
Sep 11 13:05:43.215343 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.215320    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:43.237085 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.237062    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7xqq5\""
Sep 11 13:05:43.243052 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.243029    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:43.265787 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.265764    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lg7nk\""
Sep 11 13:05:43.269020 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.268978    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9rsjp"
Sep 11 13:05:43.272555 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.272528    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-np5mv\""
Sep 11 13:05:43.274503 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.274442    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:05:43.298759 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.298728    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p5hww"]
Sep 11 13:05:43.299415 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:05:43.299264    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6942647_056e_4b32_a2b4_ea2e90f7be76.slice/crio-bed480756cda44d27af3ce7b55b7530e64c826178ddd8e78d9e67ff710e900b8 WatchSource:0}: Error finding container bed480756cda44d27af3ce7b55b7530e64c826178ddd8e78d9e67ff710e900b8: Status 404 returned error can't find the container with id bed480756cda44d27af3ce7b55b7530e64c826178ddd8e78d9e67ff710e900b8
Sep 11 13:05:43.484057 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.483979    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt"]
Sep 11 13:05:43.487140 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:05:43.487112    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e4b7f10_74fd_40fd_ba55_61080037f297.slice/crio-3490eeada0dc7ac0f707b7cea2e680c9881b1e26364fd594afcfef3ce3b9ca9f WatchSource:0}: Error finding container 3490eeada0dc7ac0f707b7cea2e680c9881b1e26364fd594afcfef3ce3b9ca9f: Status 404 returned error can't find the container with id 3490eeada0dc7ac0f707b7cea2e680c9881b1e26364fd594afcfef3ce3b9ca9f
Sep 11 13:05:43.489546 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.489502    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p5hww" event={"ID":"c6942647-056e-4b32-a2b4-ea2e90f7be76","Type":"ContainerStarted","Data":"bed480756cda44d27af3ce7b55b7530e64c826178ddd8e78d9e67ff710e900b8"}
Sep 11 13:05:43.510461 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.510430    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6548b9b645-mln9l"]
Sep 11 13:05:43.513813 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:05:43.513787    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod458a0bb9_feca_4918_a2c4_799b4659f336.slice/crio-4cf74ea711c78989ee58ec651b5855b382509122a18b6c69265cd177beb2abf9 WatchSource:0}: Error finding container 4cf74ea711c78989ee58ec651b5855b382509122a18b6c69265cd177beb2abf9: Status 404 returned error can't find the container with id 4cf74ea711c78989ee58ec651b5855b382509122a18b6c69265cd177beb2abf9
Sep 11 13:05:43.557927 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.557901    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6b946bb9b8-kxhmt"]
Sep 11 13:05:43.570391 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:05:43.570342    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda282b450_2ac0_4aed_98ac_7be36ff01407.slice/crio-63507bc6c143af951fc98712370e768476c29a18c16e1a2a7a534b02fe9d74ef WatchSource:0}: Error finding container 63507bc6c143af951fc98712370e768476c29a18c16e1a2a7a534b02fe9d74ef: Status 404 returned error can't find the container with id 63507bc6c143af951fc98712370e768476c29a18c16e1a2a7a534b02fe9d74ef
Sep 11 13:05:43.586753 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.586707    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4hb2m"]
Sep 11 13:05:43.589911 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:05:43.589875    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54af5720_e75b_4e36_94cc_4a72fb166eb4.slice/crio-578791e4e0d8fedcc63779a788c6677310d18b85114bbf712b672b9a59feb4ea WatchSource:0}: Error finding container 578791e4e0d8fedcc63779a788c6677310d18b85114bbf712b672b9a59feb4ea: Status 404 returned error can't find the container with id 578791e4e0d8fedcc63779a788c6677310d18b85114bbf712b672b9a59feb4ea
Sep 11 13:05:43.607188 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.607161    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9rsjp"]
Sep 11 13:05:43.611174 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:05:43.611137    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0498fed7_8022_4a6b_9e69_3a560c5f5e02.slice/crio-6f624731a9cfbbaa52b7cc58b22088373d215c5cf0ecf9e65710755c1851a582 WatchSource:0}: Error finding container 6f624731a9cfbbaa52b7cc58b22088373d215c5cf0ecf9e65710755c1851a582: Status 404 returned error can't find the container with id 6f624731a9cfbbaa52b7cc58b22088373d215c5cf0ecf9e65710755c1851a582
Sep 11 13:05:43.633689 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:43.633596    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qbm58"]
Sep 11 13:05:43.636882 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:05:43.636857    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod439302b6_1aa6_4530_9e4f_3534c305b8e1.slice/crio-fd0aed76f3d8d8f8d9cb3638afb517d6499074bc603de25e9ef3c13f9d172d74 WatchSource:0}: Error finding container fd0aed76f3d8d8f8d9cb3638afb517d6499074bc603de25e9ef3c13f9d172d74: Status 404 returned error can't find the container with id fd0aed76f3d8d8f8d9cb3638afb517d6499074bc603de25e9ef3c13f9d172d74
Sep 11 13:05:44.368062 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:44.368020    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:44.457231 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:44.457163    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:44.501058 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:44.500967    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6b946bb9b8-kxhmt" event={"ID":"a282b450-2ac0-4aed-98ac-7be36ff01407","Type":"ContainerStarted","Data":"4fde6bf3749d57a6fdd7305d5b5675859791c91f51002e03657732dd4d8c3199"}
Sep 11 13:05:44.501058 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:44.501024    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6b946bb9b8-kxhmt" event={"ID":"a282b450-2ac0-4aed-98ac-7be36ff01407","Type":"ContainerStarted","Data":"63507bc6c143af951fc98712370e768476c29a18c16e1a2a7a534b02fe9d74ef"}
Sep 11 13:05:44.503226 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:44.503154    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt" event={"ID":"6e4b7f10-74fd-40fd-ba55-61080037f297","Type":"ContainerStarted","Data":"3490eeada0dc7ac0f707b7cea2e680c9881b1e26364fd594afcfef3ce3b9ca9f"}
Sep 11 13:05:44.506044 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:44.506015    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qbm58" event={"ID":"439302b6-1aa6-4530-9e4f-3534c305b8e1","Type":"ContainerStarted","Data":"fd0aed76f3d8d8f8d9cb3638afb517d6499074bc603de25e9ef3c13f9d172d74"}
Sep 11 13:05:44.508966 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:44.508936    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9rsjp" event={"ID":"0498fed7-8022-4a6b-9e69-3a560c5f5e02","Type":"ContainerStarted","Data":"6f624731a9cfbbaa52b7cc58b22088373d215c5cf0ecf9e65710755c1851a582"}
Sep 11 13:05:44.510565 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:44.510530    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4hb2m" event={"ID":"54af5720-e75b-4e36-94cc-4a72fb166eb4","Type":"ContainerStarted","Data":"578791e4e0d8fedcc63779a788c6677310d18b85114bbf712b672b9a59feb4ea"}
Sep 11 13:05:44.514225 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:44.513552    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6548b9b645-mln9l" event={"ID":"458a0bb9-feca-4918-a2c4-799b4659f336","Type":"ContainerStarted","Data":"9fcd9db997ce6337bba7530748d6dcc7266a536a5a3f8e5a3a344b6d0951827d"}
Sep 11 13:05:44.514225 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:44.513603    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6548b9b645-mln9l" event={"ID":"458a0bb9-feca-4918-a2c4-799b4659f336","Type":"ContainerStarted","Data":"4cf74ea711c78989ee58ec651b5855b382509122a18b6c69265cd177beb2abf9"}
Sep 11 13:05:44.514225 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:44.514127    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:05:44.572118 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:44.570922    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6b946bb9b8-kxhmt" podStartSLOduration=37.57089635 podStartE2EDuration="37.57089635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:44.536220971 +0000 UTC m=+67.853678022" watchObservedRunningTime="2026-09-11 13:05:44.57089635 +0000 UTC m=+67.888353393"
Sep 11 13:05:45.104663 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:45.104626    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:45.206534 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:45.206443    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:45.216344 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:45.216300    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:45.219533 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:45.219508    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:45.244952 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:45.244568    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6548b9b645-mln9l" podStartSLOduration=68.244546382 podStartE2EDuration="1m8.244546382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:44.571604061 +0000 UTC m=+67.889061102" watchObservedRunningTime="2026-09-11 13:05:45.244546382 +0000 UTC m=+68.562003423"
Sep 11 13:05:45.525010 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:45.524970    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9rsjp" event={"ID":"0498fed7-8022-4a6b-9e69-3a560c5f5e02","Type":"ContainerStarted","Data":"984ead6eef138445a41064aa26a70d1ba95c35ddac4dada140737b11b23c6111"}
Sep 11 13:05:45.525541 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:45.525511    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9rsjp" event={"ID":"0498fed7-8022-4a6b-9e69-3a560c5f5e02","Type":"ContainerStarted","Data":"078cb092fa49d2a3fc898e7a6c9a549bbb06aeb6e858d4b102380b7a2adf795f"}
Sep 11 13:05:45.525626 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:45.525552    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:45.526977 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:45.526953    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6b946bb9b8-kxhmt"
Sep 11 13:05:45.548416 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:45.548343    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9rsjp" podStartSLOduration=67.406975921 podStartE2EDuration="1m8.548322382s" totalImagesPullingTime="1.141346461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:45.542339949 +0000 UTC m=+68.859796990" watchObservedRunningTime="2026-09-11 13:05:45.548322382 +0000 UTC m=+68.865779423"
Sep 11 13:05:45.689853 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:45.689816    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:45.810756 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:45.810663    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:46.151971 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:46.151929    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:46.249668 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:46.249633    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:47.925782 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:47.925737    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:48.006883 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:48.006845    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:48.508672 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:48.508635    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:48.584600 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:48.584559    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:49.318806 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.318750    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:49.459614 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.459577    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:49.542742 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.542710    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt" event={"ID":"6e4b7f10-74fd-40fd-ba55-61080037f297","Type":"ContainerStarted","Data":"834154968c7d6d2992f54709a6990157b3b744830349b9a2692c832b743c06c6"}
Sep 11 13:05:49.544259 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.544228    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p5hww" event={"ID":"c6942647-056e-4b32-a2b4-ea2e90f7be76","Type":"ContainerStarted","Data":"a2734f1d9953aa27cee58db43272c7d8cebd083533a9ae6c8f926461b2268913"}
Sep 11 13:05:49.545820 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.545792    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qbm58" event={"ID":"439302b6-1aa6-4530-9e4f-3534c305b8e1","Type":"ContainerStarted","Data":"8294a9b702ea53c9469e33a2c58d9844f29c99042c1d1c05a5272c1cb09b1b48"}
Sep 11 13:05:49.545945 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.545866    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:05:49.547638 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.547614    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4hb2m" event={"ID":"54af5720-e75b-4e36-94cc-4a72fb166eb4","Type":"ContainerStarted","Data":"3c4db57771af3c486996ec5b1abccb8ad5f69565e02781a25a2bde57162ade70"}
Sep 11 13:05:49.547750 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.547648    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4hb2m" event={"ID":"54af5720-e75b-4e36-94cc-4a72fb166eb4","Type":"ContainerStarted","Data":"d1b54b9cb7c69e27b3deab00cc440b82c49c51c94406730309d0b3a6d11247bc"}
Sep 11 13:05:49.547814 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.547748    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:05:49.647456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.647325    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:49.647598 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.647490    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p5hww" podStartSLOduration=67.011652372 podStartE2EDuration="1m12.647473469s" totalImagesPullingTime="5.635821097s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:49.647347576 +0000 UTC m=+72.964804617" watchObservedRunningTime="2026-09-11 13:05:49.647473469 +0000 UTC m=+72.964930508"
Sep 11 13:05:49.647734 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.647713    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4rqbt" podStartSLOduration=36.211040034 podStartE2EDuration="41.647707484s" totalImagesPullingTime="5.43666745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:49.603490173 +0000 UTC m=+72.920947213" watchObservedRunningTime="2026-09-11 13:05:49.647707484 +0000 UTC m=+72.965164556"
Sep 11 13:05:49.672999 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.672945    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qbm58" podStartSLOduration=67.379710668 podStartE2EDuration="1m12.672927316s" totalImagesPullingTime="5.293216648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:49.670932132 +0000 UTC m=+72.988389173" watchObservedRunningTime="2026-09-11 13:05:49.672927316 +0000 UTC m=+72.990384367"
Sep 11 13:05:49.712782 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.712687    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4hb2m" podStartSLOduration=33.378773454 podStartE2EDuration="38.712665213s" totalImagesPullingTime="5.333891759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:49.712092088 +0000 UTC m=+73.029549128" watchObservedRunningTime="2026-09-11 13:05:49.712665213 +0000 UTC m=+73.030122254"
Sep 11 13:05:49.812994 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:49.812942    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:51.276626 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:51.276583    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:51.279030 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:51.279002    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/663c3e69-ad92-4794-b0c4-9b8c319b176c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mp69k\" (UID: \"663c3e69-ad92-4794-b0c4-9b8c319b176c\") " pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:51.485874 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:51.485843    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jddd5\""
Sep 11 13:05:51.494190 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:51.494165    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mp69k"
Sep 11 13:05:51.794615 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:51.794524    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mp69k"]
Sep 11 13:05:51.798053 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:05:51.798019    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod663c3e69_ad92_4794_b0c4_9b8c319b176c.slice/crio-51f16db6067d28fa5c9e0517e8ac959d505ddd4ccc79ca390827d01ebc08ceda WatchSource:0}: Error finding container 51f16db6067d28fa5c9e0517e8ac959d505ddd4ccc79ca390827d01ebc08ceda: Status 404 returned error can't find the container with id 51f16db6067d28fa5c9e0517e8ac959d505ddd4ccc79ca390827d01ebc08ceda
Sep 11 13:05:52.560662 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:52.560623    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mp69k" event={"ID":"663c3e69-ad92-4794-b0c4-9b8c319b176c","Type":"ContainerStarted","Data":"302e4d141b7cb0d3a8b58c40a01d5a632becde56c4a3044bcfb80d4db5781b36"}
Sep 11 13:05:52.561057 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:52.560689    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mp69k" event={"ID":"663c3e69-ad92-4794-b0c4-9b8c319b176c","Type":"ContainerStarted","Data":"51f16db6067d28fa5c9e0517e8ac959d505ddd4ccc79ca390827d01ebc08ceda"}
Sep 11 13:05:53.190931 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:53.190860    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:53.296241 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:53.296207    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:53.566545 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:53.566517    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mp69k" event={"ID":"663c3e69-ad92-4794-b0c4-9b8c319b176c","Type":"ContainerStarted","Data":"5aedf1aa14b44bbf21bf4eb5f0110d2237aded2d19197e6f6acc01b12a50ea69"}
Sep 11 13:05:54.572662 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:54.572625    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mp69k" event={"ID":"663c3e69-ad92-4794-b0c4-9b8c319b176c","Type":"ContainerStarted","Data":"8cc4340e3771ca95a8d83dca47e570115940dee57450b5bf3c29da65374ffee3"}
Sep 11 13:05:54.590751 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:54.590705    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:54.596197 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:54.596148    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mp69k" podStartSLOduration=33.248963937 podStartE2EDuration="35.596134366s" totalImagesPullingTime="2.347170429s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:05:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:54.595109929 +0000 UTC m=+77.912566969" watchObservedRunningTime="2026-09-11 13:05:54.596134366 +0000 UTC m=+77.913591410"
Sep 11 13:05:54.691487 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:54.691393    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:56.736925 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:56.736892    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:57.549542 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:57.549505    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:57.650256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:57.650220    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:05:59.554510 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:05:59.554404    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4hb2m"
Sep 11 13:06:00.744540 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:00.744499    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:00.890587 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:00.890547    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:01.685249 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.685216    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:01.778954 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.778918    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"]
Sep 11 13:06:01.781346 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.781329    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:01.785471 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.785443    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 11 13:06:01.785599 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.785509    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-ztxcq\""
Sep 11 13:06:01.786169 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.786153    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 11 13:06:01.788946 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.788927    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 13:06:01.798520 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.798494    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"]
Sep 11 13:06:01.848511 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.848481    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:01.956056 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.955964    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3165da23-31f2-4da8-80d6-90d815044bfa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-9sbhl\" (UID: \"3165da23-31f2-4da8-80d6-90d815044bfa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:01.956056 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.956010    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh58l\" (UniqueName: \"kubernetes.io/projected/3165da23-31f2-4da8-80d6-90d815044bfa-kube-api-access-rh58l\") pod \"prometheus-operator-68bcd7f987-9sbhl\" (UID: \"3165da23-31f2-4da8-80d6-90d815044bfa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:01.956255 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.956090    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3165da23-31f2-4da8-80d6-90d815044bfa-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-9sbhl\" (UID: \"3165da23-31f2-4da8-80d6-90d815044bfa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:01.956255 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.956153    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3165da23-31f2-4da8-80d6-90d815044bfa-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-9sbhl\" (UID: \"3165da23-31f2-4da8-80d6-90d815044bfa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:01.978583 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:01.978555    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:02.056661 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.056620    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3165da23-31f2-4da8-80d6-90d815044bfa-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-9sbhl\" (UID: \"3165da23-31f2-4da8-80d6-90d815044bfa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:02.056833 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.056679    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3165da23-31f2-4da8-80d6-90d815044bfa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-9sbhl\" (UID: \"3165da23-31f2-4da8-80d6-90d815044bfa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:02.056833 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.056707    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh58l\" (UniqueName: \"kubernetes.io/projected/3165da23-31f2-4da8-80d6-90d815044bfa-kube-api-access-rh58l\") pod \"prometheus-operator-68bcd7f987-9sbhl\" (UID: \"3165da23-31f2-4da8-80d6-90d815044bfa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:02.056833 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.056748    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3165da23-31f2-4da8-80d6-90d815044bfa-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-9sbhl\" (UID: \"3165da23-31f2-4da8-80d6-90d815044bfa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:02.057439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.057414    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3165da23-31f2-4da8-80d6-90d815044bfa-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-9sbhl\" (UID: \"3165da23-31f2-4da8-80d6-90d815044bfa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:02.059126 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.059098    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3165da23-31f2-4da8-80d6-90d815044bfa-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-9sbhl\" (UID: \"3165da23-31f2-4da8-80d6-90d815044bfa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:02.059218 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.059194    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3165da23-31f2-4da8-80d6-90d815044bfa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-9sbhl\" (UID: \"3165da23-31f2-4da8-80d6-90d815044bfa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:02.066620 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.066599    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh58l\" (UniqueName: \"kubernetes.io/projected/3165da23-31f2-4da8-80d6-90d815044bfa-kube-api-access-rh58l\") pod \"prometheus-operator-68bcd7f987-9sbhl\" (UID: \"3165da23-31f2-4da8-80d6-90d815044bfa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:02.090559 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.090527    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"
Sep 11 13:06:02.387660 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.387601    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl"]
Sep 11 13:06:02.389634 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:06:02.389606    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3165da23_31f2_4da8_80d6_90d815044bfa.slice/crio-fb15b2a574e207f1c17f8ded30c146a5bee5cd8831bde42278e2776ce37a7d79 WatchSource:0}: Error finding container fb15b2a574e207f1c17f8ded30c146a5bee5cd8831bde42278e2776ce37a7d79: Status 404 returned error can't find the container with id fb15b2a574e207f1c17f8ded30c146a5bee5cd8831bde42278e2776ce37a7d79
Sep 11 13:06:02.397989 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.397959    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:02.569609 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.569578    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:02.606112 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.606081    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl" event={"ID":"3165da23-31f2-4da8-80d6-90d815044bfa","Type":"ContainerStarted","Data":"fb15b2a574e207f1c17f8ded30c146a5bee5cd8831bde42278e2776ce37a7d79"}
Sep 11 13:06:02.975953 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:02.975910    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:03.070680 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:03.070638    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:04.054990 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:04.054895    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6548b9b645-mln9l"]
Sep 11 13:06:04.060245 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:04.060219    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:06:04.068160 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:04.068131    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:04.202091 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:04.202057    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:04.300480 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:04.300446    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:04.615157 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:04.615124    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl" event={"ID":"3165da23-31f2-4da8-80d6-90d815044bfa","Type":"ContainerStarted","Data":"990c14d0c8569afa157497c8a281d2d337fda21eabfe3729794618711132f302"}
Sep 11 13:06:04.615337 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:04.615176    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl" event={"ID":"3165da23-31f2-4da8-80d6-90d815044bfa","Type":"ContainerStarted","Data":"30d12e68c34f91b629f60edb3e8c6d110574400a6cb2ce8448ca123b67f65a1c"}
Sep 11 13:06:04.635640 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:04.635535    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-9sbhl" podStartSLOduration=2.314027999 podStartE2EDuration="3.635514148s" totalImagesPullingTime="1.321486149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:04.634451737 +0000 UTC m=+87.951908806" watchObservedRunningTime="2026-09-11 13:06:04.635514148 +0000 UTC m=+87.952971558"
Sep 11 13:06:06.139989 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.139948    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:06.353202 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.353175    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:06.500282 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.500202    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"]
Sep 11 13:06:06.503673 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.503655    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.507163 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.507140    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 13:06:06.508194 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.508172    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 13:06:06.508253 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.508172    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6c7qh\""
Sep 11 13:06:06.510697 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.510645    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:06.537624 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.537593    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"]
Sep 11 13:06:06.595487 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.595431    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e6cd0e1-fff0-44de-84dc-9fb32fb6c962-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-xnhqn\" (UID: \"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.595699 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.595501    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4drn\" (UniqueName: \"kubernetes.io/projected/7e6cd0e1-fff0-44de-84dc-9fb32fb6c962-kube-api-access-r4drn\") pod \"openshift-state-metrics-67dcf8c979-xnhqn\" (UID: \"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.595699 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.595618    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e6cd0e1-fff0-44de-84dc-9fb32fb6c962-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-xnhqn\" (UID: \"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.595699 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.595662    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e6cd0e1-fff0-44de-84dc-9fb32fb6c962-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-xnhqn\" (UID: \"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.595855 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.595761    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"]
Sep 11 13:06:06.599199 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.599180    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.602072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.602051    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-q988s\""
Sep 11 13:06:06.602592 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.602573    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 11 13:06:06.608342 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.608322    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 11 13:06:06.608506 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.608424    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 11 13:06:06.609437 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.609408    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-77bk5"]
Sep 11 13:06:06.612623 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.612607    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.615481 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.615465    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 13:06:06.615548 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.615486    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 13:06:06.616085 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.616058    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 13:06:06.618505 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.618485    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6lgh2\""
Sep 11 13:06:06.625220 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.625198    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"]
Sep 11 13:06:06.696837 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.696802    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e6cd0e1-fff0-44de-84dc-9fb32fb6c962-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-xnhqn\" (UID: \"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.697005 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.696848    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4drn\" (UniqueName: \"kubernetes.io/projected/7e6cd0e1-fff0-44de-84dc-9fb32fb6c962-kube-api-access-r4drn\") pod \"openshift-state-metrics-67dcf8c979-xnhqn\" (UID: \"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.697005 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.696880    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/453275be-2adc-4e7b-aed8-cf4e83a6165b-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.697005 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.696915    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/453275be-2adc-4e7b-aed8-cf4e83a6165b-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.697005 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.696939    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/453275be-2adc-4e7b-aed8-cf4e83a6165b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.697005 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.696973    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e6cd0e1-fff0-44de-84dc-9fb32fb6c962-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-xnhqn\" (UID: \"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.697203 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.697038    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/453275be-2adc-4e7b-aed8-cf4e83a6165b-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.697203 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.697091    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e6cd0e1-fff0-44de-84dc-9fb32fb6c962-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-xnhqn\" (UID: \"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.697203 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.697129    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmcjx\" (UniqueName: \"kubernetes.io/projected/453275be-2adc-4e7b-aed8-cf4e83a6165b-kube-api-access-tmcjx\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.697313 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.697197    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/453275be-2adc-4e7b-aed8-cf4e83a6165b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.697741 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.697723    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e6cd0e1-fff0-44de-84dc-9fb32fb6c962-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-xnhqn\" (UID: \"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.699303 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.699285    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e6cd0e1-fff0-44de-84dc-9fb32fb6c962-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-xnhqn\" (UID: \"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.699397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.699289    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e6cd0e1-fff0-44de-84dc-9fb32fb6c962-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-xnhqn\" (UID: \"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.708610 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.708585    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4drn\" (UniqueName: \"kubernetes.io/projected/7e6cd0e1-fff0-44de-84dc-9fb32fb6c962-kube-api-access-r4drn\") pod \"openshift-state-metrics-67dcf8c979-xnhqn\" (UID: \"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.753722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.753626    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:06.798562 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798526    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-tls\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.798694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798571    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.798694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798606    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/453275be-2adc-4e7b-aed8-cf4e83a6165b-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.798694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798641    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/453275be-2adc-4e7b-aed8-cf4e83a6165b-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.798694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798665    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/453275be-2adc-4e7b-aed8-cf4e83a6165b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.798694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798691    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13816e24-f2f6-49d4-8f05-d3f5c23adc94-sys\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.798944 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798711    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfmqv\" (UniqueName: \"kubernetes.io/projected/13816e24-f2f6-49d4-8f05-d3f5c23adc94-kube-api-access-vfmqv\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.798944 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798741    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-textfile\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.798944 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798762    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13816e24-f2f6-49d4-8f05-d3f5c23adc94-metrics-client-ca\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.798944 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798811    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-accelerators-collector-config\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.798944 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798858    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/453275be-2adc-4e7b-aed8-cf4e83a6165b-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.798944 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798892    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-wtmp\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.798944 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798941    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmcjx\" (UniqueName: \"kubernetes.io/projected/453275be-2adc-4e7b-aed8-cf4e83a6165b-kube-api-access-tmcjx\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.799293 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798966    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/13816e24-f2f6-49d4-8f05-d3f5c23adc94-root\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.799293 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.798997    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/453275be-2adc-4e7b-aed8-cf4e83a6165b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.799293 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.799167    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/453275be-2adc-4e7b-aed8-cf4e83a6165b-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.799470 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.799451    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/453275be-2adc-4e7b-aed8-cf4e83a6165b-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.799798 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.799770    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/453275be-2adc-4e7b-aed8-cf4e83a6165b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.801169 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.801139    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/453275be-2adc-4e7b-aed8-cf4e83a6165b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.801280 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.801211    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/453275be-2adc-4e7b-aed8-cf4e83a6165b-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.812150 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.812125    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"
Sep 11 13:06:06.820976 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.820950    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmcjx\" (UniqueName: \"kubernetes.io/projected/453275be-2adc-4e7b-aed8-cf4e83a6165b-kube-api-access-tmcjx\") pod \"kube-state-metrics-59fbdd77d-qmw72\" (UID: \"453275be-2adc-4e7b-aed8-cf4e83a6165b\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.900167 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900134    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-tls\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.900167 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900166    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.900397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900194    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13816e24-f2f6-49d4-8f05-d3f5c23adc94-sys\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.900397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900213    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfmqv\" (UniqueName: \"kubernetes.io/projected/13816e24-f2f6-49d4-8f05-d3f5c23adc94-kube-api-access-vfmqv\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.900397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900231    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-textfile\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.900397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900245    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13816e24-f2f6-49d4-8f05-d3f5c23adc94-metrics-client-ca\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.900397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900264    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-accelerators-collector-config\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.900397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900284    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-wtmp\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.900397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900310    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/13816e24-f2f6-49d4-8f05-d3f5c23adc94-root\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.900397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900345    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13816e24-f2f6-49d4-8f05-d3f5c23adc94-sys\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.900751 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900429    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/13816e24-f2f6-49d4-8f05-d3f5c23adc94-root\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.900751 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900540    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-wtmp\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.900751 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900695    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-textfile\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.901000 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900979    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13816e24-f2f6-49d4-8f05-d3f5c23adc94-metrics-client-ca\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.901000 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.900992    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-accelerators-collector-config\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.902515 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.902497    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-tls\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.903005 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.902988    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13816e24-f2f6-49d4-8f05-d3f5c23adc94-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:06.908134 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.908110    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"
Sep 11 13:06:06.920876 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.920853    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:06.922042 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:06.922020    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfmqv\" (UniqueName: \"kubernetes.io/projected/13816e24-f2f6-49d4-8f05-d3f5c23adc94-kube-api-access-vfmqv\") pod \"node-exporter-77bk5\" (UID: \"13816e24-f2f6-49d4-8f05-d3f5c23adc94\") " pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:07.072116 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:07.072089    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:07.104150 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:07.104014    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn"]
Sep 11 13:06:07.106069 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:06:07.106019    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6cd0e1_fff0_44de_84dc_9fb32fb6c962.slice/crio-165a08e0320b48caa2e617e7a562cda20e039ef5ebaa13b1d6d84e70279e8043 WatchSource:0}: Error finding container 165a08e0320b48caa2e617e7a562cda20e039ef5ebaa13b1d6d84e70279e8043: Status 404 returned error can't find the container with id 165a08e0320b48caa2e617e7a562cda20e039ef5ebaa13b1d6d84e70279e8043
Sep 11 13:06:07.192792 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:07.192767    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72"]
Sep 11 13:06:07.196317 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:06:07.196262    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod453275be_2adc_4e7b_aed8_cf4e83a6165b.slice/crio-3cd59969c6121491108147ca3acc1dd6568333af88111c72aa86bcedc97c851c WatchSource:0}: Error finding container 3cd59969c6121491108147ca3acc1dd6568333af88111c72aa86bcedc97c851c: Status 404 returned error can't find the container with id 3cd59969c6121491108147ca3acc1dd6568333af88111c72aa86bcedc97c851c
Sep 11 13:06:07.220610 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:07.220584    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-77bk5"
Sep 11 13:06:07.232306 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:06:07.232229    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13816e24_f2f6_49d4_8f05_d3f5c23adc94.slice/crio-9930f62148b07f486b1c009b53a11842d5570f2d41719f6e70993cbc1dbcc650 WatchSource:0}: Error finding container 9930f62148b07f486b1c009b53a11842d5570f2d41719f6e70993cbc1dbcc650: Status 404 returned error can't find the container with id 9930f62148b07f486b1c009b53a11842d5570f2d41719f6e70993cbc1dbcc650
Sep 11 13:06:07.293042 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:07.293004    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:07.421276 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:07.421184    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:07.587481 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:07.587443    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:07.628221 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:07.628186    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72" event={"ID":"453275be-2adc-4e7b-aed8-cf4e83a6165b","Type":"ContainerStarted","Data":"3cd59969c6121491108147ca3acc1dd6568333af88111c72aa86bcedc97c851c"}
Sep 11 13:06:07.630102 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:07.630069    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn" event={"ID":"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962","Type":"ContainerStarted","Data":"7b34e22d574d7340c99380f475feefa5aae168783dbc2aafa531d27f0b60b400"}
Sep 11 13:06:07.630226 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:07.630132    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn" event={"ID":"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962","Type":"ContainerStarted","Data":"a086b6e6fcdd4555817aff4b6a5d7e4c1668e30e74b68fc37a0d78ec30710435"}
Sep 11 13:06:07.630226 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:07.630160    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn" event={"ID":"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962","Type":"ContainerStarted","Data":"165a08e0320b48caa2e617e7a562cda20e039ef5ebaa13b1d6d84e70279e8043"}
Sep 11 13:06:07.631301 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:07.631277    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-77bk5" event={"ID":"13816e24-f2f6-49d4-8f05-d3f5c23adc94","Type":"ContainerStarted","Data":"9930f62148b07f486b1c009b53a11842d5570f2d41719f6e70993cbc1dbcc650"}
Sep 11 13:06:07.779734 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:07.779652    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:08.818405 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:08.818374    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:08.922622 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:08.922592    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:09.394558 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:09.394508    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:09.642274 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:09.642240    2554 generic.go:356] "Generic (PLEG): container finished" podID="13816e24-f2f6-49d4-8f05-d3f5c23adc94" containerID="e9f656e2eda943228f01088f077da7a9b2c546b758e98b52de7ec9af9ef012a2" exitCode=0
Sep 11 13:06:09.642499 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:09.642335    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-77bk5" event={"ID":"13816e24-f2f6-49d4-8f05-d3f5c23adc94","Type":"ContainerDied","Data":"e9f656e2eda943228f01088f077da7a9b2c546b758e98b52de7ec9af9ef012a2"}
Sep 11 13:06:09.645399 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:09.645343    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72" event={"ID":"453275be-2adc-4e7b-aed8-cf4e83a6165b","Type":"ContainerStarted","Data":"a6fd3c080704ef5c9a6722bf877d3d47bece1087149a39d2328b2c71960b23c4"}
Sep 11 13:06:09.645526 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:09.645432    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72" event={"ID":"453275be-2adc-4e7b-aed8-cf4e83a6165b","Type":"ContainerStarted","Data":"02e13fec8237ae57cfd3b02f180fe3d026dc80a2001da05916c0277e90996aec"}
Sep 11 13:06:09.645526 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:09.645461    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72" event={"ID":"453275be-2adc-4e7b-aed8-cf4e83a6165b","Type":"ContainerStarted","Data":"720a7b80f7583b08b2f9d7bd362c3d08f60375aa69fc7249721fdf4915ba81e0"}
Sep 11 13:06:09.649485 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:09.649462    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn" event={"ID":"7e6cd0e1-fff0-44de-84dc-9fb32fb6c962","Type":"ContainerStarted","Data":"b465258f6a2772f44961e8774866015a64371a644dd01a0df077602958a6a8dd"}
Sep 11 13:06:09.691286 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:09.691229    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-xnhqn" podStartSLOduration=2.074079282 podStartE2EDuration="3.691207385s" totalImagesPullingTime="1.617128103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:09.69101527 +0000 UTC m=+93.008472323" watchObservedRunningTime="2026-09-11 13:06:09.691207385 +0000 UTC m=+93.008664424"
Sep 11 13:06:09.768148 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:09.768114    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:09.869811 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:09.869776    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:09.991270 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:09.991186    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:10.358464 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:10.358435    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:10.459279 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:10.459248    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:10.656038 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:10.655934    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-77bk5" event={"ID":"13816e24-f2f6-49d4-8f05-d3f5c23adc94","Type":"ContainerStarted","Data":"df9a4a08204a5cdd0daec901fe761fe20db2dfca42c0f32baa14257b579fc370"}
Sep 11 13:06:10.656038 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:10.656030    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-77bk5" event={"ID":"13816e24-f2f6-49d4-8f05-d3f5c23adc94","Type":"ContainerStarted","Data":"47667fc494528e2986c5f1326664f959e13e2b2777e86e04bbd512407549e200"}
Sep 11 13:06:10.677347 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:10.677289    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-77bk5" podStartSLOduration=2.977940599 podStartE2EDuration="4.677271518s" totalImagesPullingTime="1.621277072s" totalInitContainerRuntime="78.053847ms" isStatefulPod=true podCreationTimestamp="2026-09-11 13:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:10.67661458 +0000 UTC m=+93.994071619" watchObservedRunningTime="2026-09-11 13:06:10.677271518 +0000 UTC m=+93.994728559"
Sep 11 13:06:10.678289 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:10.678254    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-qmw72" podStartSLOduration=3.019867457 podStartE2EDuration="4.678242957s" totalImagesPullingTime="1.6583755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:09.718021428 +0000 UTC m=+93.035478483" watchObservedRunningTime="2026-09-11 13:06:10.678242957 +0000 UTC m=+93.995699996"
Sep 11 13:06:10.983058 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:10.982972    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:11.011382 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.011329    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"]
Sep 11 13:06:11.014642 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.014625    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"
Sep 11 13:06:11.017236 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.017211    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2wb6x\""
Sep 11 13:06:11.017375 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.017250    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 11 13:06:11.029078 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.029051    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"]
Sep 11 13:06:11.035802 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.035775    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frm89\" (UniqueName: \"kubernetes.io/projected/67d490a6-167c-4a9c-9d94-e5ca6eb44cd0-kube-api-access-frm89\") pod \"monitoring-plugin-5d85778b9d-sp56g\" (UID: \"67d490a6-167c-4a9c-9d94-e5ca6eb44cd0\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"
Sep 11 13:06:11.035955 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.035843    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67d490a6-167c-4a9c-9d94-e5ca6eb44cd0-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-sp56g\" (UID: \"67d490a6-167c-4a9c-9d94-e5ca6eb44cd0\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"
Sep 11 13:06:11.076382 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.076337    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:11.136620 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.136576    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67d490a6-167c-4a9c-9d94-e5ca6eb44cd0-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-sp56g\" (UID: \"67d490a6-167c-4a9c-9d94-e5ca6eb44cd0\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"
Sep 11 13:06:11.136794 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.136665    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frm89\" (UniqueName: \"kubernetes.io/projected/67d490a6-167c-4a9c-9d94-e5ca6eb44cd0-kube-api-access-frm89\") pod \"monitoring-plugin-5d85778b9d-sp56g\" (UID: \"67d490a6-167c-4a9c-9d94-e5ca6eb44cd0\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"
Sep 11 13:06:11.136794 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:06:11.136729    2554 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 11 13:06:11.136891 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:06:11.136795    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67d490a6-167c-4a9c-9d94-e5ca6eb44cd0-monitoring-plugin-cert podName:67d490a6-167c-4a9c-9d94-e5ca6eb44cd0 nodeName:}" failed. No retries permitted until 2026-09-11 13:06:11.63677829 +0000 UTC m=+94.954235308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/67d490a6-167c-4a9c-9d94-e5ca6eb44cd0-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-sp56g" (UID: "67d490a6-167c-4a9c-9d94-e5ca6eb44cd0") : secret "monitoring-plugin-cert" not found
Sep 11 13:06:11.151784 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.151751    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frm89\" (UniqueName: \"kubernetes.io/projected/67d490a6-167c-4a9c-9d94-e5ca6eb44cd0-kube-api-access-frm89\") pod \"monitoring-plugin-5d85778b9d-sp56g\" (UID: \"67d490a6-167c-4a9c-9d94-e5ca6eb44cd0\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"
Sep 11 13:06:11.227173 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.227140    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:11.642084 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.642049    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67d490a6-167c-4a9c-9d94-e5ca6eb44cd0-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-sp56g\" (UID: \"67d490a6-167c-4a9c-9d94-e5ca6eb44cd0\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"
Sep 11 13:06:11.644585 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.644560    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67d490a6-167c-4a9c-9d94-e5ca6eb44cd0-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-sp56g\" (UID: \"67d490a6-167c-4a9c-9d94-e5ca6eb44cd0\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"
Sep 11 13:06:11.924976 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.924891    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"
Sep 11 13:06:11.998341 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:11.998311    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:12.099968 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:12.099931    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:12.201860 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:12.201832    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"]
Sep 11 13:06:12.204389 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:06:12.204349    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67d490a6_167c_4a9c_9d94_e5ca6eb44cd0.slice/crio-4ee084217814209ee14d40784cb29f07b199fb955fba94df2c512e8f45f478f3 WatchSource:0}: Error finding container 4ee084217814209ee14d40784cb29f07b199fb955fba94df2c512e8f45f478f3: Status 404 returned error can't find the container with id 4ee084217814209ee14d40784cb29f07b199fb955fba94df2c512e8f45f478f3
Sep 11 13:06:12.664931 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:12.664896    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g" event={"ID":"67d490a6-167c-4a9c-9d94-e5ca6eb44cd0","Type":"ContainerStarted","Data":"4ee084217814209ee14d40784cb29f07b199fb955fba94df2c512e8f45f478f3"}
Sep 11 13:06:13.087449 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:13.087407    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:13.331168 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:13.331132    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:13.529002 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:13.528915    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:14.527459 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:14.527423    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:14.669733 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:14.669700    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:14.675128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:14.675100    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g" event={"ID":"67d490a6-167c-4a9c-9d94-e5ca6eb44cd0","Type":"ContainerStarted","Data":"0ac607d6c4aa5b2dd12f23bd8b577ac69cc5e06b91a24e3420df39c78fd3e80c"}
Sep 11 13:06:14.675305 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:14.675291    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"
Sep 11 13:06:14.680424 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:14.680397    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g"
Sep 11 13:06:14.695435 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:14.695388    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-sp56g" podStartSLOduration=3.077753385 podStartE2EDuration="4.695358785s" totalImagesPullingTime="1.6176054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:14.694967142 +0000 UTC m=+98.012424183" watchObservedRunningTime="2026-09-11 13:06:14.695358785 +0000 UTC m=+98.012815880"
Sep 11 13:06:14.768191 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:14.768154    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:15.273026 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:15.272996    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:15.374285 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:15.374248    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:17.919359 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:17.919321    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:18.112772 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:18.112735    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:20.553883 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:20.553852    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qbm58"
Sep 11 13:06:27.648919 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:27.648879    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:29.075502 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.075439    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6548b9b645-mln9l" podUID="458a0bb9-feca-4918-a2c4-799b4659f336" containerName="registry" containerID="cri-o://9fcd9db997ce6337bba7530748d6dcc7266a536a5a3f8e5a3a344b6d0951827d" gracePeriod=30
Sep 11 13:06:29.306771 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.306749    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:06:29.374043 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.373956    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/458a0bb9-feca-4918-a2c4-799b4659f336-trusted-ca\") pod \"458a0bb9-feca-4918-a2c4-799b4659f336\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") "
Sep 11 13:06:29.374043 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.374008    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4m5b6\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-kube-api-access-4m5b6\") pod \"458a0bb9-feca-4918-a2c4-799b4659f336\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") "
Sep 11 13:06:29.374043 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.374043    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls\") pod \"458a0bb9-feca-4918-a2c4-799b4659f336\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") "
Sep 11 13:06:29.374283 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.374071    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/458a0bb9-feca-4918-a2c4-799b4659f336-ca-trust-extracted\") pod \"458a0bb9-feca-4918-a2c4-799b4659f336\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") "
Sep 11 13:06:29.374283 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.374129    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/458a0bb9-feca-4918-a2c4-799b4659f336-image-registry-private-configuration\") pod \"458a0bb9-feca-4918-a2c4-799b4659f336\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") "
Sep 11 13:06:29.374283 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.374171    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/458a0bb9-feca-4918-a2c4-799b4659f336-registry-certificates\") pod \"458a0bb9-feca-4918-a2c4-799b4659f336\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") "
Sep 11 13:06:29.374283 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.374206    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-bound-sa-token\") pod \"458a0bb9-feca-4918-a2c4-799b4659f336\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") "
Sep 11 13:06:29.374283 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.374233    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/458a0bb9-feca-4918-a2c4-799b4659f336-installation-pull-secrets\") pod \"458a0bb9-feca-4918-a2c4-799b4659f336\" (UID: \"458a0bb9-feca-4918-a2c4-799b4659f336\") "
Sep 11 13:06:29.374567 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.374483    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/458a0bb9-feca-4918-a2c4-799b4659f336-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "458a0bb9-feca-4918-a2c4-799b4659f336" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:29.375047 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.374971    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/458a0bb9-feca-4918-a2c4-799b4659f336-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "458a0bb9-feca-4918-a2c4-799b4659f336" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:29.376788 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.376759    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "458a0bb9-feca-4918-a2c4-799b4659f336" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:06:29.376918 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.376821    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "458a0bb9-feca-4918-a2c4-799b4659f336" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:06:29.376918 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.376907    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-kube-api-access-4m5b6" (OuterVolumeSpecName: "kube-api-access-4m5b6") pod "458a0bb9-feca-4918-a2c4-799b4659f336" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336"). InnerVolumeSpecName "kube-api-access-4m5b6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:06:29.377033 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.377013    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/458a0bb9-feca-4918-a2c4-799b4659f336-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "458a0bb9-feca-4918-a2c4-799b4659f336" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:06:29.377113 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.377088    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/458a0bb9-feca-4918-a2c4-799b4659f336-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "458a0bb9-feca-4918-a2c4-799b4659f336" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:06:29.382600 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.382574    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/458a0bb9-feca-4918-a2c4-799b4659f336-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "458a0bb9-feca-4918-a2c4-799b4659f336" (UID: "458a0bb9-feca-4918-a2c4-799b4659f336"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:06:29.475389 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.475322    2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/458a0bb9-feca-4918-a2c4-799b4659f336-trusted-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:06:29.475389 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.475360    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4m5b6\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-kube-api-access-4m5b6\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:06:29.475389 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.475394    2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-registry-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:06:29.475389 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.475403    2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/458a0bb9-feca-4918-a2c4-799b4659f336-ca-trust-extracted\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:06:29.475649 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.475413    2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/458a0bb9-feca-4918-a2c4-799b4659f336-image-registry-private-configuration\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:06:29.475649 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.475423    2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/458a0bb9-feca-4918-a2c4-799b4659f336-registry-certificates\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:06:29.475649 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.475431    2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/458a0bb9-feca-4918-a2c4-799b4659f336-bound-sa-token\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:06:29.475649 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.475439    2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/458a0bb9-feca-4918-a2c4-799b4659f336-installation-pull-secrets\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:06:29.732050 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.731968    2554 generic.go:356] "Generic (PLEG): container finished" podID="458a0bb9-feca-4918-a2c4-799b4659f336" containerID="9fcd9db997ce6337bba7530748d6dcc7266a536a5a3f8e5a3a344b6d0951827d" exitCode=0
Sep 11 13:06:29.732050 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.732024    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6548b9b645-mln9l"
Sep 11 13:06:29.732215 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.732053    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6548b9b645-mln9l" event={"ID":"458a0bb9-feca-4918-a2c4-799b4659f336","Type":"ContainerDied","Data":"9fcd9db997ce6337bba7530748d6dcc7266a536a5a3f8e5a3a344b6d0951827d"}
Sep 11 13:06:29.732215 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.732108    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6548b9b645-mln9l" event={"ID":"458a0bb9-feca-4918-a2c4-799b4659f336","Type":"ContainerDied","Data":"4cf74ea711c78989ee58ec651b5855b382509122a18b6c69265cd177beb2abf9"}
Sep 11 13:06:29.732215 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.732128    2554 scope.go:117] "RemoveContainer" containerID="9fcd9db997ce6337bba7530748d6dcc7266a536a5a3f8e5a3a344b6d0951827d"
Sep 11 13:06:29.740910 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.740892    2554 scope.go:117] "RemoveContainer" containerID="9fcd9db997ce6337bba7530748d6dcc7266a536a5a3f8e5a3a344b6d0951827d"
Sep 11 13:06:29.741172 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:06:29.741152    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fcd9db997ce6337bba7530748d6dcc7266a536a5a3f8e5a3a344b6d0951827d\": container with ID starting with 9fcd9db997ce6337bba7530748d6dcc7266a536a5a3f8e5a3a344b6d0951827d not found: ID does not exist" containerID="9fcd9db997ce6337bba7530748d6dcc7266a536a5a3f8e5a3a344b6d0951827d"
Sep 11 13:06:29.741224 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.741180    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fcd9db997ce6337bba7530748d6dcc7266a536a5a3f8e5a3a344b6d0951827d"} err="failed to get container status \"9fcd9db997ce6337bba7530748d6dcc7266a536a5a3f8e5a3a344b6d0951827d\": rpc error: code = NotFound desc = could not find container \"9fcd9db997ce6337bba7530748d6dcc7266a536a5a3f8e5a3a344b6d0951827d\": container with ID starting with 9fcd9db997ce6337bba7530748d6dcc7266a536a5a3f8e5a3a344b6d0951827d not found: ID does not exist"
Sep 11 13:06:29.754094 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.754069    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6548b9b645-mln9l"]
Sep 11 13:06:29.756216 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:29.756196    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6548b9b645-mln9l"]
Sep 11 13:06:30.479681 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:30.479648    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:30.586557 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:30.586522    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:30.755217 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:30.755139    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:31.152557 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:31.152524    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="458a0bb9-feca-4918-a2c4-799b4659f336" path="/var/lib/kubelet/pods/458a0bb9-feca-4918-a2c4-799b4659f336/volumes"
Sep 11 13:06:32.975704 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:32.975664    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:33.047435 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:33.047407    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:06:39.769066 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:39.769029    2554 generic.go:356] "Generic (PLEG): container finished" podID="cbe4efbd-bb09-403a-a183-f0efd34611ee" containerID="f7092f796ba765d20ed383e813e161f2781819e8baa81f2f5df5237d3c3a630a" exitCode=0
Sep 11 13:06:39.769438 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:39.769105    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7" event={"ID":"cbe4efbd-bb09-403a-a183-f0efd34611ee","Type":"ContainerDied","Data":"f7092f796ba765d20ed383e813e161f2781819e8baa81f2f5df5237d3c3a630a"}
Sep 11 13:06:39.769507 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:39.769446    2554 scope.go:117] "RemoveContainer" containerID="f7092f796ba765d20ed383e813e161f2781819e8baa81f2f5df5237d3c3a630a"
Sep 11 13:06:40.775076 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:40.775036    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-b65r7" event={"ID":"cbe4efbd-bb09-403a-a183-f0efd34611ee","Type":"ContainerStarted","Data":"99434fed5a791da67d1e2c921dcfe1c22ed103c7cbfb9f5fdb945386df711ab6"}
Sep 11 13:06:43.788692 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:43.788602    2554 generic.go:356] "Generic (PLEG): container finished" podID="7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e" containerID="5fd092ac842beb12aaba4a6cec2ff80f2255c3775a457e90caf8b9eb6ea3d767" exitCode=0
Sep 11 13:06:43.788692 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:43.788652    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-p49pp" event={"ID":"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e","Type":"ContainerDied","Data":"5fd092ac842beb12aaba4a6cec2ff80f2255c3775a457e90caf8b9eb6ea3d767"}
Sep 11 13:06:43.789068 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:43.789019    2554 scope.go:117] "RemoveContainer" containerID="5fd092ac842beb12aaba4a6cec2ff80f2255c3775a457e90caf8b9eb6ea3d767"
Sep 11 13:06:44.794382 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:06:44.794328    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-p49pp" event={"ID":"7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e","Type":"ContainerStarted","Data":"5fdf842345a68927ad568f1c5b939073358cabb5fb3dbe914c79d7541027c4a3"}
Sep 11 13:07:22.952669 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:07:22.952635    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:07:23.034107 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:07:23.034063    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:07:31.945469 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:07:31.945389    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:07:32.068874 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:07:32.068837    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:07:35.028325 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:07:35.028289    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:07:35.130171 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:07:35.130133    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:07:42.908051 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:07:42.908018    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:07:43.036471 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:07:43.036438    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:07:48.037797 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:07:48.037724    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:07:48.175813 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:07:48.175780    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:08:37.156828 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:08:37.156794    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:08:37.257139 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:08:37.257105    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:08:45.158180 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:08:45.158126    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:09:37.057212 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:09:37.057182    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:09:37.062586 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:09:37.062562    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:09:37.068105 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:09:37.068081    2554 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 13:09:43.390812 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:09:43.390766    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:18.079174 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.079143    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"]
Sep 11 13:10:18.079671 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.079432    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="458a0bb9-feca-4918-a2c4-799b4659f336" containerName="registry"
Sep 11 13:10:18.079671 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.079444    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="458a0bb9-feca-4918-a2c4-799b4659f336" containerName="registry"
Sep 11 13:10:18.079671 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.079504    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="458a0bb9-feca-4918-a2c4-799b4659f336" containerName="registry"
Sep 11 13:10:18.082522 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.082505    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"
Sep 11 13:10:18.094823 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.094787    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:10:18.095646 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.095629    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:10:18.095713 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.095631    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-r6hnf\""
Sep 11 13:10:18.130875 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.130842    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"]
Sep 11 13:10:18.153623 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.153588    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c626a155-ccbe-4047-b946-910183c57e51-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b\" (UID: \"c626a155-ccbe-4047-b946-910183c57e51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"
Sep 11 13:10:18.153796 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.153644    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mmqv\" (UniqueName: \"kubernetes.io/projected/c626a155-ccbe-4047-b946-910183c57e51-kube-api-access-7mmqv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b\" (UID: \"c626a155-ccbe-4047-b946-910183c57e51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"
Sep 11 13:10:18.153796 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.153667    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c626a155-ccbe-4047-b946-910183c57e51-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b\" (UID: \"c626a155-ccbe-4047-b946-910183c57e51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"
Sep 11 13:10:18.254281 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.254247    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c626a155-ccbe-4047-b946-910183c57e51-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b\" (UID: \"c626a155-ccbe-4047-b946-910183c57e51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"
Sep 11 13:10:18.254449 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.254297    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mmqv\" (UniqueName: \"kubernetes.io/projected/c626a155-ccbe-4047-b946-910183c57e51-kube-api-access-7mmqv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b\" (UID: \"c626a155-ccbe-4047-b946-910183c57e51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"
Sep 11 13:10:18.254449 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.254318    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c626a155-ccbe-4047-b946-910183c57e51-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b\" (UID: \"c626a155-ccbe-4047-b946-910183c57e51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"
Sep 11 13:10:18.254666 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.254647    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c626a155-ccbe-4047-b946-910183c57e51-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b\" (UID: \"c626a155-ccbe-4047-b946-910183c57e51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"
Sep 11 13:10:18.254746 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.254709    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c626a155-ccbe-4047-b946-910183c57e51-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b\" (UID: \"c626a155-ccbe-4047-b946-910183c57e51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"
Sep 11 13:10:18.264789 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.264756    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mmqv\" (UniqueName: \"kubernetes.io/projected/c626a155-ccbe-4047-b946-910183c57e51-kube-api-access-7mmqv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b\" (UID: \"c626a155-ccbe-4047-b946-910183c57e51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"
Sep 11 13:10:18.391215 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.391121    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"
Sep 11 13:10:18.670123 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.670044    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"]
Sep 11 13:10:18.673233 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:10:18.673207    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc626a155_ccbe_4047_b946_910183c57e51.slice/crio-fca7b2553433039940d18260616c688488b76329a53b20e2ccf10f736c96e731 WatchSource:0}: Error finding container fca7b2553433039940d18260616c688488b76329a53b20e2ccf10f736c96e731: Status 404 returned error can't find the container with id fca7b2553433039940d18260616c688488b76329a53b20e2ccf10f736c96e731
Sep 11 13:10:18.675124 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:18.675105    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:10:19.589983 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:19.589945    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b" event={"ID":"c626a155-ccbe-4047-b946-910183c57e51","Type":"ContainerStarted","Data":"fca7b2553433039940d18260616c688488b76329a53b20e2ccf10f736c96e731"}
Sep 11 13:10:24.611923 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:24.611885    2554 generic.go:356] "Generic (PLEG): container finished" podID="c626a155-ccbe-4047-b946-910183c57e51" containerID="36d407613fa73edc47f363a46c47455a77da27258399944c8909eeb5637fff2a" exitCode=0
Sep 11 13:10:24.612294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:24.611953    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b" event={"ID":"c626a155-ccbe-4047-b946-910183c57e51","Type":"ContainerDied","Data":"36d407613fa73edc47f363a46c47455a77da27258399944c8909eeb5637fff2a"}
Sep 11 13:10:26.623170 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:26.623135    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b" event={"ID":"c626a155-ccbe-4047-b946-910183c57e51","Type":"ContainerStarted","Data":"408b8e300d1b4ff5ba8619a434490adff38f0de01f7fdc68786fab658dabd255"}
Sep 11 13:10:27.628542 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:27.628506    2554 generic.go:356] "Generic (PLEG): container finished" podID="c626a155-ccbe-4047-b946-910183c57e51" containerID="408b8e300d1b4ff5ba8619a434490adff38f0de01f7fdc68786fab658dabd255" exitCode=0
Sep 11 13:10:27.628542 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:27.628546    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b" event={"ID":"c626a155-ccbe-4047-b946-910183c57e51","Type":"ContainerDied","Data":"408b8e300d1b4ff5ba8619a434490adff38f0de01f7fdc68786fab658dabd255"}
Sep 11 13:10:31.758175 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:31.758085    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:34.660342 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:34.660309    2554 generic.go:356] "Generic (PLEG): container finished" podID="c626a155-ccbe-4047-b946-910183c57e51" containerID="d760669e1b5f942ea7527b265853450e44df0cebd277b9e09ee0946c4c788760" exitCode=0
Sep 11 13:10:34.660745 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:34.660382    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b" event={"ID":"c626a155-ccbe-4047-b946-910183c57e51","Type":"ContainerDied","Data":"d760669e1b5f942ea7527b265853450e44df0cebd277b9e09ee0946c4c788760"}
Sep 11 13:10:35.781336 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:35.781312    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"
Sep 11 13:10:35.905277 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:35.905234    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c626a155-ccbe-4047-b946-910183c57e51-bundle\") pod \"c626a155-ccbe-4047-b946-910183c57e51\" (UID: \"c626a155-ccbe-4047-b946-910183c57e51\") "
Sep 11 13:10:35.905497 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:35.905295    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mmqv\" (UniqueName: \"kubernetes.io/projected/c626a155-ccbe-4047-b946-910183c57e51-kube-api-access-7mmqv\") pod \"c626a155-ccbe-4047-b946-910183c57e51\" (UID: \"c626a155-ccbe-4047-b946-910183c57e51\") "
Sep 11 13:10:35.905497 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:35.905332    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c626a155-ccbe-4047-b946-910183c57e51-util\") pod \"c626a155-ccbe-4047-b946-910183c57e51\" (UID: \"c626a155-ccbe-4047-b946-910183c57e51\") "
Sep 11 13:10:35.905952 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:35.905925    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c626a155-ccbe-4047-b946-910183c57e51-bundle" (OuterVolumeSpecName: "bundle") pod "c626a155-ccbe-4047-b946-910183c57e51" (UID: "c626a155-ccbe-4047-b946-910183c57e51"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:10:35.907459 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:35.907436    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c626a155-ccbe-4047-b946-910183c57e51-kube-api-access-7mmqv" (OuterVolumeSpecName: "kube-api-access-7mmqv") pod "c626a155-ccbe-4047-b946-910183c57e51" (UID: "c626a155-ccbe-4047-b946-910183c57e51"). InnerVolumeSpecName "kube-api-access-7mmqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:10:35.909684 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:35.909656    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c626a155-ccbe-4047-b946-910183c57e51-util" (OuterVolumeSpecName: "util") pod "c626a155-ccbe-4047-b946-910183c57e51" (UID: "c626a155-ccbe-4047-b946-910183c57e51"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:10:36.006712 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:36.006615    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mmqv\" (UniqueName: \"kubernetes.io/projected/c626a155-ccbe-4047-b946-910183c57e51-kube-api-access-7mmqv\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:10:36.006712 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:36.006650    2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c626a155-ccbe-4047-b946-910183c57e51-util\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:10:36.006712 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:36.006661    2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c626a155-ccbe-4047-b946-910183c57e51-bundle\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:10:36.669352 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:36.669322    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b"
Sep 11 13:10:36.669352 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:36.669337    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lwb2b" event={"ID":"c626a155-ccbe-4047-b946-910183c57e51","Type":"ContainerDied","Data":"fca7b2553433039940d18260616c688488b76329a53b20e2ccf10f736c96e731"}
Sep 11 13:10:36.669579 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:36.669402    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fca7b2553433039940d18260616c688488b76329a53b20e2ccf10f736c96e731"
Sep 11 13:10:39.539346 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:39.539310    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:39.921182 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:39.921138    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:41.928687 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.928649    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc"]
Sep 11 13:10:41.929061 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.928973    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c626a155-ccbe-4047-b946-910183c57e51" containerName="util"
Sep 11 13:10:41.929061 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.928985    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c626a155-ccbe-4047-b946-910183c57e51" containerName="util"
Sep 11 13:10:41.929061 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.928993    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c626a155-ccbe-4047-b946-910183c57e51" containerName="extract"
Sep 11 13:10:41.929061 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.929001    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c626a155-ccbe-4047-b946-910183c57e51" containerName="extract"
Sep 11 13:10:41.929061 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.929017    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c626a155-ccbe-4047-b946-910183c57e51" containerName="pull"
Sep 11 13:10:41.929061 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.929023    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c626a155-ccbe-4047-b946-910183c57e51" containerName="pull"
Sep 11 13:10:41.929242 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.929067    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c626a155-ccbe-4047-b946-910183c57e51" containerName="extract"
Sep 11 13:10:41.931026 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.931010    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc"
Sep 11 13:10:41.933834 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.933811    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 11 13:10:41.933987 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.933903    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:10:41.933987 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.933919    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-9rlhc\""
Sep 11 13:10:41.942772 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.942745    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc"]
Sep 11 13:10:41.958654 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.958618    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1861dfb6-0e49-406e-b5ca-eae41c3bf613-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-q7glc\" (UID: \"1861dfb6-0e49-406e-b5ca-eae41c3bf613\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc"
Sep 11 13:10:41.958794 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:41.958691    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sh26\" (UniqueName: \"kubernetes.io/projected/1861dfb6-0e49-406e-b5ca-eae41c3bf613-kube-api-access-6sh26\") pod \"cert-manager-operator-controller-manager-55c444bf86-q7glc\" (UID: \"1861dfb6-0e49-406e-b5ca-eae41c3bf613\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc"
Sep 11 13:10:42.059897 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:42.059857    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sh26\" (UniqueName: \"kubernetes.io/projected/1861dfb6-0e49-406e-b5ca-eae41c3bf613-kube-api-access-6sh26\") pod \"cert-manager-operator-controller-manager-55c444bf86-q7glc\" (UID: \"1861dfb6-0e49-406e-b5ca-eae41c3bf613\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc"
Sep 11 13:10:42.060050 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:42.060014    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1861dfb6-0e49-406e-b5ca-eae41c3bf613-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-q7glc\" (UID: \"1861dfb6-0e49-406e-b5ca-eae41c3bf613\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc"
Sep 11 13:10:42.060355 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:42.060301    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1861dfb6-0e49-406e-b5ca-eae41c3bf613-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-q7glc\" (UID: \"1861dfb6-0e49-406e-b5ca-eae41c3bf613\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc"
Sep 11 13:10:42.069275 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:42.069246    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sh26\" (UniqueName: \"kubernetes.io/projected/1861dfb6-0e49-406e-b5ca-eae41c3bf613-kube-api-access-6sh26\") pod \"cert-manager-operator-controller-manager-55c444bf86-q7glc\" (UID: \"1861dfb6-0e49-406e-b5ca-eae41c3bf613\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc"
Sep 11 13:10:42.240710 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:42.240618    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc"
Sep 11 13:10:42.527191 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:42.527091    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc"]
Sep 11 13:10:42.531260 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:10:42.531229    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1861dfb6_0e49_406e_b5ca_eae41c3bf613.slice/crio-0ddd9329e2e195b8a4fd2136fffed30d9a38db1c1f42aedc8093b6e3400f0cad WatchSource:0}: Error finding container 0ddd9329e2e195b8a4fd2136fffed30d9a38db1c1f42aedc8093b6e3400f0cad: Status 404 returned error can't find the container with id 0ddd9329e2e195b8a4fd2136fffed30d9a38db1c1f42aedc8093b6e3400f0cad
Sep 11 13:10:42.695552 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:42.695512    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc" event={"ID":"1861dfb6-0e49-406e-b5ca-eae41c3bf613","Type":"ContainerStarted","Data":"0ddd9329e2e195b8a4fd2136fffed30d9a38db1c1f42aedc8093b6e3400f0cad"}
Sep 11 13:10:43.521181 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:43.521144    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:43.638311 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:43.638270    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:44.706890 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:44.706849    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc" event={"ID":"1861dfb6-0e49-406e-b5ca-eae41c3bf613","Type":"ContainerStarted","Data":"69dcd7c94600f42d218c1bf52a103ac440eb6dc82b4b98b7478ce54f3a9c87ed"}
Sep 11 13:10:44.729568 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:44.729504    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q7glc" podStartSLOduration=2.063639872 podStartE2EDuration="3.72948268s" totalImagesPullingTime="1.665842808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:10:44.72694903 +0000 UTC m=+368.044406069" watchObservedRunningTime="2026-09-11 13:10:44.72948268 +0000 UTC m=+368.046939725"
Sep 11 13:10:45.034627 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.034544    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:45.149104 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.149076    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:45.253550 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.253520    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n42d8"]
Sep 11 13:10:45.255719 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.255701    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-n42d8"
Sep 11 13:10:45.259409 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.259390    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-sn5sg\""
Sep 11 13:10:45.260694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.260675    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 13:10:45.260796 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.260679    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 13:10:45.263635 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.263612    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:45.270784 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.270761    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n42d8"]
Sep 11 13:10:45.287552 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.287462    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e3cacad-6d32-4f35-82b3-6f918b121eae-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n42d8\" (UID: \"1e3cacad-6d32-4f35-82b3-6f918b121eae\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n42d8"
Sep 11 13:10:45.287552 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.287524    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k679\" (UniqueName: \"kubernetes.io/projected/1e3cacad-6d32-4f35-82b3-6f918b121eae-kube-api-access-9k679\") pod \"cert-manager-webhook-6486ffd997-n42d8\" (UID: \"1e3cacad-6d32-4f35-82b3-6f918b121eae\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n42d8"
Sep 11 13:10:45.388458 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.388419    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e3cacad-6d32-4f35-82b3-6f918b121eae-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n42d8\" (UID: \"1e3cacad-6d32-4f35-82b3-6f918b121eae\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n42d8"
Sep 11 13:10:45.388599 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.388483    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k679\" (UniqueName: \"kubernetes.io/projected/1e3cacad-6d32-4f35-82b3-6f918b121eae-kube-api-access-9k679\") pod \"cert-manager-webhook-6486ffd997-n42d8\" (UID: \"1e3cacad-6d32-4f35-82b3-6f918b121eae\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n42d8"
Sep 11 13:10:45.410807 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.410773    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k679\" (UniqueName: \"kubernetes.io/projected/1e3cacad-6d32-4f35-82b3-6f918b121eae-kube-api-access-9k679\") pod \"cert-manager-webhook-6486ffd997-n42d8\" (UID: \"1e3cacad-6d32-4f35-82b3-6f918b121eae\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n42d8"
Sep 11 13:10:45.414496 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.414469    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e3cacad-6d32-4f35-82b3-6f918b121eae-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n42d8\" (UID: \"1e3cacad-6d32-4f35-82b3-6f918b121eae\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n42d8"
Sep 11 13:10:45.575958 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.575928    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-n42d8"
Sep 11 13:10:45.854569 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.854491    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n42d8"]
Sep 11 13:10:45.858512 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:10:45.858485    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e3cacad_6d32_4f35_82b3_6f918b121eae.slice/crio-9fa788fe07929655660fca23c23c657d08f690a5fb172fb41a717ee4039ed3c9 WatchSource:0}: Error finding container 9fa788fe07929655660fca23c23c657d08f690a5fb172fb41a717ee4039ed3c9: Status 404 returned error can't find the container with id 9fa788fe07929655660fca23c23c657d08f690a5fb172fb41a717ee4039ed3c9
Sep 11 13:10:45.885465 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.885432    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"]
Sep 11 13:10:45.888470 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.888448    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"
Sep 11 13:10:45.891255 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.891233    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:10:45.891356 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.891235    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-r6hnf\""
Sep 11 13:10:45.891356 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.891235    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:10:45.908845 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.908818    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"]
Sep 11 13:10:45.993134 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.993097    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b08ad25-317e-43c6-a0a2-7c185188a083-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7\" (UID: \"9b08ad25-317e-43c6-a0a2-7c185188a083\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"
Sep 11 13:10:45.993315 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.993147    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4cmt\" (UniqueName: \"kubernetes.io/projected/9b08ad25-317e-43c6-a0a2-7c185188a083-kube-api-access-z4cmt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7\" (UID: \"9b08ad25-317e-43c6-a0a2-7c185188a083\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"
Sep 11 13:10:45.993315 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:45.993218    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b08ad25-317e-43c6-a0a2-7c185188a083-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7\" (UID: \"9b08ad25-317e-43c6-a0a2-7c185188a083\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"
Sep 11 13:10:46.094625 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:46.094585    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b08ad25-317e-43c6-a0a2-7c185188a083-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7\" (UID: \"9b08ad25-317e-43c6-a0a2-7c185188a083\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"
Sep 11 13:10:46.094817 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:46.094644    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4cmt\" (UniqueName: \"kubernetes.io/projected/9b08ad25-317e-43c6-a0a2-7c185188a083-kube-api-access-z4cmt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7\" (UID: \"9b08ad25-317e-43c6-a0a2-7c185188a083\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"
Sep 11 13:10:46.094817 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:46.094697    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b08ad25-317e-43c6-a0a2-7c185188a083-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7\" (UID: \"9b08ad25-317e-43c6-a0a2-7c185188a083\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"
Sep 11 13:10:46.095085 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:46.095062    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b08ad25-317e-43c6-a0a2-7c185188a083-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7\" (UID: \"9b08ad25-317e-43c6-a0a2-7c185188a083\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"
Sep 11 13:10:46.095196 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:46.095062    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b08ad25-317e-43c6-a0a2-7c185188a083-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7\" (UID: \"9b08ad25-317e-43c6-a0a2-7c185188a083\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"
Sep 11 13:10:46.105324 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:46.105275    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4cmt\" (UniqueName: \"kubernetes.io/projected/9b08ad25-317e-43c6-a0a2-7c185188a083-kube-api-access-z4cmt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7\" (UID: \"9b08ad25-317e-43c6-a0a2-7c185188a083\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"
Sep 11 13:10:46.197918 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:46.197879    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"
Sep 11 13:10:46.482888 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:46.482861    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"]
Sep 11 13:10:46.485446 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:10:46.485417    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b08ad25_317e_43c6_a0a2_7c185188a083.slice/crio-34fb1bee924c62ca5d127b7caccbd1e22bed6366278ae6c2ad164a909ea3afbb WatchSource:0}: Error finding container 34fb1bee924c62ca5d127b7caccbd1e22bed6366278ae6c2ad164a909ea3afbb: Status 404 returned error can't find the container with id 34fb1bee924c62ca5d127b7caccbd1e22bed6366278ae6c2ad164a909ea3afbb
Sep 11 13:10:46.718392 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:46.718342    2554 generic.go:356] "Generic (PLEG): container finished" podID="9b08ad25-317e-43c6-a0a2-7c185188a083" containerID="fd3bf17598c2446b5837285bcc9124f5e71dc299b1555153b783647e46994234" exitCode=0
Sep 11 13:10:46.718575 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:46.718433    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7" event={"ID":"9b08ad25-317e-43c6-a0a2-7c185188a083","Type":"ContainerDied","Data":"fd3bf17598c2446b5837285bcc9124f5e71dc299b1555153b783647e46994234"}
Sep 11 13:10:46.718575 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:46.718496    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7" event={"ID":"9b08ad25-317e-43c6-a0a2-7c185188a083","Type":"ContainerStarted","Data":"34fb1bee924c62ca5d127b7caccbd1e22bed6366278ae6c2ad164a909ea3afbb"}
Sep 11 13:10:46.720020 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:46.719981    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-n42d8" event={"ID":"1e3cacad-6d32-4f35-82b3-6f918b121eae","Type":"ContainerStarted","Data":"9fa788fe07929655660fca23c23c657d08f690a5fb172fb41a717ee4039ed3c9"}
Sep 11 13:10:46.969811 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:46.969777    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:47.267753 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:47.267671    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:47.646278 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:47.646246    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:49.737711 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:49.737618    2554 generic.go:356] "Generic (PLEG): container finished" podID="9b08ad25-317e-43c6-a0a2-7c185188a083" containerID="c1bd214defc2bed255531b554a89cf7bd5355aea2aa0794bf28dc134c2743956" exitCode=0
Sep 11 13:10:49.737711 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:49.737698    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7" event={"ID":"9b08ad25-317e-43c6-a0a2-7c185188a083","Type":"ContainerDied","Data":"c1bd214defc2bed255531b554a89cf7bd5355aea2aa0794bf28dc134c2743956"}
Sep 11 13:10:49.739428 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:49.739406    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-n42d8" event={"ID":"1e3cacad-6d32-4f35-82b3-6f918b121eae","Type":"ContainerStarted","Data":"0c8f238b4ec6c4ece708b0af34ac7cd405ad916176ded21142bbf63e9a282f45"}
Sep 11 13:10:49.739579 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:49.739525    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-n42d8"
Sep 11 13:10:49.775345 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:49.775174    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-n42d8" podStartSLOduration=1.309751796 podStartE2EDuration="4.775153801s" totalImagesPullingTime="3.465402005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:10:49.774222912 +0000 UTC m=+373.091680068" watchObservedRunningTime="2026-09-11 13:10:49.775153801 +0000 UTC m=+373.092610841"
Sep 11 13:10:50.099431 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:50.099393    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:50.204469 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:50.204431    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:50.746297 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:50.746265    2554 generic.go:356] "Generic (PLEG): container finished" podID="9b08ad25-317e-43c6-a0a2-7c185188a083" containerID="0bfeb5725083db54582d1c13689c499531fcb68602633abbc4679c760a1c1a8b" exitCode=0
Sep 11 13:10:50.746720 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:50.746359    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7" event={"ID":"9b08ad25-317e-43c6-a0a2-7c185188a083","Type":"ContainerDied","Data":"0bfeb5725083db54582d1c13689c499531fcb68602633abbc4679c760a1c1a8b"}
Sep 11 13:10:51.880042 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:51.880014    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"
Sep 11 13:10:51.947104 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:51.947072    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4cmt\" (UniqueName: \"kubernetes.io/projected/9b08ad25-317e-43c6-a0a2-7c185188a083-kube-api-access-z4cmt\") pod \"9b08ad25-317e-43c6-a0a2-7c185188a083\" (UID: \"9b08ad25-317e-43c6-a0a2-7c185188a083\") "
Sep 11 13:10:51.947265 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:51.947113    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b08ad25-317e-43c6-a0a2-7c185188a083-bundle\") pod \"9b08ad25-317e-43c6-a0a2-7c185188a083\" (UID: \"9b08ad25-317e-43c6-a0a2-7c185188a083\") "
Sep 11 13:10:51.947265 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:51.947177    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b08ad25-317e-43c6-a0a2-7c185188a083-util\") pod \"9b08ad25-317e-43c6-a0a2-7c185188a083\" (UID: \"9b08ad25-317e-43c6-a0a2-7c185188a083\") "
Sep 11 13:10:51.947561 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:51.947528    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b08ad25-317e-43c6-a0a2-7c185188a083-bundle" (OuterVolumeSpecName: "bundle") pod "9b08ad25-317e-43c6-a0a2-7c185188a083" (UID: "9b08ad25-317e-43c6-a0a2-7c185188a083"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:10:51.949178 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:51.949152    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b08ad25-317e-43c6-a0a2-7c185188a083-kube-api-access-z4cmt" (OuterVolumeSpecName: "kube-api-access-z4cmt") pod "9b08ad25-317e-43c6-a0a2-7c185188a083" (UID: "9b08ad25-317e-43c6-a0a2-7c185188a083"). InnerVolumeSpecName "kube-api-access-z4cmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:10:51.953713 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:51.953671    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b08ad25-317e-43c6-a0a2-7c185188a083-util" (OuterVolumeSpecName: "util") pod "9b08ad25-317e-43c6-a0a2-7c185188a083" (UID: "9b08ad25-317e-43c6-a0a2-7c185188a083"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:10:52.048201 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:52.048108    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4cmt\" (UniqueName: \"kubernetes.io/projected/9b08ad25-317e-43c6-a0a2-7c185188a083-kube-api-access-z4cmt\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:10:52.048201 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:52.048142    2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b08ad25-317e-43c6-a0a2-7c185188a083-bundle\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:10:52.048201 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:52.048151    2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b08ad25-317e-43c6-a0a2-7c185188a083-util\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:10:52.756909 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:52.756877    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7" event={"ID":"9b08ad25-317e-43c6-a0a2-7c185188a083","Type":"ContainerDied","Data":"34fb1bee924c62ca5d127b7caccbd1e22bed6366278ae6c2ad164a909ea3afbb"}
Sep 11 13:10:52.756909 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:52.756906    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fj9bx7"
Sep 11 13:10:52.757113 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:52.756928    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34fb1bee924c62ca5d127b7caccbd1e22bed6366278ae6c2ad164a909ea3afbb"
Sep 11 13:10:55.016746 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:55.016713    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:55.748969 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:55.748936    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-n42d8"
Sep 11 13:10:56.542123 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.542083    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2hctm"]
Sep 11 13:10:56.542607 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.542509    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b08ad25-317e-43c6-a0a2-7c185188a083" containerName="pull"
Sep 11 13:10:56.542607 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.542527    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b08ad25-317e-43c6-a0a2-7c185188a083" containerName="pull"
Sep 11 13:10:56.542607 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.542549    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b08ad25-317e-43c6-a0a2-7c185188a083" containerName="util"
Sep 11 13:10:56.542607 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.542557    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b08ad25-317e-43c6-a0a2-7c185188a083" containerName="util"
Sep 11 13:10:56.542607 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.542582    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b08ad25-317e-43c6-a0a2-7c185188a083" containerName="extract"
Sep 11 13:10:56.542607 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.542590    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b08ad25-317e-43c6-a0a2-7c185188a083" containerName="extract"
Sep 11 13:10:56.542942 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.542664    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b08ad25-317e-43c6-a0a2-7c185188a083" containerName="extract"
Sep 11 13:10:56.546393 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.546352    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2hctm"
Sep 11 13:10:56.550198 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.550175    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-qqg9c\""
Sep 11 13:10:56.557895 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.557869    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2hctm"]
Sep 11 13:10:56.590539 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.590504    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b4296f88-884f-45f3-942b-1b5765f67115-bound-sa-token\") pod \"cert-manager-55b4d954d5-2hctm\" (UID: \"b4296f88-884f-45f3-942b-1b5765f67115\") " pod="cert-manager/cert-manager-55b4d954d5-2hctm"
Sep 11 13:10:56.590539 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.590542    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqxn7\" (UniqueName: \"kubernetes.io/projected/b4296f88-884f-45f3-942b-1b5765f67115-kube-api-access-xqxn7\") pod \"cert-manager-55b4d954d5-2hctm\" (UID: \"b4296f88-884f-45f3-942b-1b5765f67115\") " pod="cert-manager/cert-manager-55b4d954d5-2hctm"
Sep 11 13:10:56.691924 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.691884    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b4296f88-884f-45f3-942b-1b5765f67115-bound-sa-token\") pod \"cert-manager-55b4d954d5-2hctm\" (UID: \"b4296f88-884f-45f3-942b-1b5765f67115\") " pod="cert-manager/cert-manager-55b4d954d5-2hctm"
Sep 11 13:10:56.691924 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.691925    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqxn7\" (UniqueName: \"kubernetes.io/projected/b4296f88-884f-45f3-942b-1b5765f67115-kube-api-access-xqxn7\") pod \"cert-manager-55b4d954d5-2hctm\" (UID: \"b4296f88-884f-45f3-942b-1b5765f67115\") " pod="cert-manager/cert-manager-55b4d954d5-2hctm"
Sep 11 13:10:56.702031 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.701997    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b4296f88-884f-45f3-942b-1b5765f67115-bound-sa-token\") pod \"cert-manager-55b4d954d5-2hctm\" (UID: \"b4296f88-884f-45f3-942b-1b5765f67115\") " pod="cert-manager/cert-manager-55b4d954d5-2hctm"
Sep 11 13:10:56.704095 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.704072    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqxn7\" (UniqueName: \"kubernetes.io/projected/b4296f88-884f-45f3-942b-1b5765f67115-kube-api-access-xqxn7\") pod \"cert-manager-55b4d954d5-2hctm\" (UID: \"b4296f88-884f-45f3-942b-1b5765f67115\") " pod="cert-manager/cert-manager-55b4d954d5-2hctm"
Sep 11 13:10:56.855321 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:56.855286    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2hctm"
Sep 11 13:10:57.132944 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:57.132868    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2hctm"]
Sep 11 13:10:57.135998 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:10:57.135968    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4296f88_884f_45f3_942b_1b5765f67115.slice/crio-1a88a2506bda7ba961679ca9c58254d75a278623808cc5fe335da254e0744109 WatchSource:0}: Error finding container 1a88a2506bda7ba961679ca9c58254d75a278623808cc5fe335da254e0744109: Status 404 returned error can't find the container with id 1a88a2506bda7ba961679ca9c58254d75a278623808cc5fe335da254e0744109
Sep 11 13:10:57.329032 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:57.328995    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:57.430750 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:57.430679    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:57.618177 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:57.618143    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:57.727914 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:57.727840    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:57.779074 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:57.779029    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2hctm" event={"ID":"b4296f88-884f-45f3-942b-1b5765f67115","Type":"ContainerStarted","Data":"6407ab67906b28890f53113d46698811e9e73f63c2d877be059f779700132799"}
Sep 11 13:10:57.779074 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:57.779075    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2hctm" event={"ID":"b4296f88-884f-45f3-942b-1b5765f67115","Type":"ContainerStarted","Data":"1a88a2506bda7ba961679ca9c58254d75a278623808cc5fe335da254e0744109"}
Sep 11 13:10:57.814580 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:57.814524    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-2hctm" podStartSLOduration=1.8145026340000001 podStartE2EDuration="1.814502634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:10:57.812405951 +0000 UTC m=+381.129862989" watchObservedRunningTime="2026-09-11 13:10:57.814502634 +0000 UTC m=+381.131959677"
Sep 11 13:10:57.817722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:57.817701    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:10:58.895329 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:58.895300    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb"]
Sep 11 13:10:58.932053 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:58.932015    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb"]
Sep 11 13:10:58.932208 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:58.932108    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb"
Sep 11 13:10:58.934779 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:58.934760    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:10:58.934868 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:58.934761    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-5jgx5\""
Sep 11 13:10:58.935273 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:58.935255    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 13:10:59.011668 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:59.011628    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76b6a8a8-c009-4818-a7cb-8a2b77a82e57-tmp\") pod \"openshift-lws-operator-fd8ccff4c-t6bxb\" (UID: \"76b6a8a8-c009-4818-a7cb-8a2b77a82e57\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb"
Sep 11 13:10:59.011831 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:59.011698    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdr5x\" (UniqueName: \"kubernetes.io/projected/76b6a8a8-c009-4818-a7cb-8a2b77a82e57-kube-api-access-xdr5x\") pod \"openshift-lws-operator-fd8ccff4c-t6bxb\" (UID: \"76b6a8a8-c009-4818-a7cb-8a2b77a82e57\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb"
Sep 11 13:10:59.112657 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:59.112616    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76b6a8a8-c009-4818-a7cb-8a2b77a82e57-tmp\") pod \"openshift-lws-operator-fd8ccff4c-t6bxb\" (UID: \"76b6a8a8-c009-4818-a7cb-8a2b77a82e57\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb"
Sep 11 13:10:59.112828 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:59.112703    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdr5x\" (UniqueName: \"kubernetes.io/projected/76b6a8a8-c009-4818-a7cb-8a2b77a82e57-kube-api-access-xdr5x\") pod \"openshift-lws-operator-fd8ccff4c-t6bxb\" (UID: \"76b6a8a8-c009-4818-a7cb-8a2b77a82e57\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb"
Sep 11 13:10:59.113012 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:59.112991    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76b6a8a8-c009-4818-a7cb-8a2b77a82e57-tmp\") pod \"openshift-lws-operator-fd8ccff4c-t6bxb\" (UID: \"76b6a8a8-c009-4818-a7cb-8a2b77a82e57\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb"
Sep 11 13:10:59.126706 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:59.126673    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdr5x\" (UniqueName: \"kubernetes.io/projected/76b6a8a8-c009-4818-a7cb-8a2b77a82e57-kube-api-access-xdr5x\") pod \"openshift-lws-operator-fd8ccff4c-t6bxb\" (UID: \"76b6a8a8-c009-4818-a7cb-8a2b77a82e57\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb"
Sep 11 13:10:59.241518 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:59.241425    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb"
Sep 11 13:10:59.526059 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:59.525943    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb"]
Sep 11 13:10:59.528629 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:10:59.528602    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76b6a8a8_c009_4818_a7cb_8a2b77a82e57.slice/crio-5548d7393c14e88f6d2497fde40ba13bfe6af95ecbbd205d0573651cf96fdd8e WatchSource:0}: Error finding container 5548d7393c14e88f6d2497fde40ba13bfe6af95ecbbd205d0573651cf96fdd8e: Status 404 returned error can't find the container with id 5548d7393c14e88f6d2497fde40ba13bfe6af95ecbbd205d0573651cf96fdd8e
Sep 11 13:10:59.789122 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:10:59.789045    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb" event={"ID":"76b6a8a8-c009-4818-a7cb-8a2b77a82e57","Type":"ContainerStarted","Data":"5548d7393c14e88f6d2497fde40ba13bfe6af95ecbbd205d0573651cf96fdd8e"}
Sep 11 13:11:02.802896 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:02.802862    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb" event={"ID":"76b6a8a8-c009-4818-a7cb-8a2b77a82e57","Type":"ContainerStarted","Data":"33d401f7028482cd77a67f6f976ca5190f965fde92c65c6697ce050779601661"}
Sep 11 13:11:02.821738 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:02.821687    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-t6bxb" podStartSLOduration=2.634087017 podStartE2EDuration="4.821671728s" totalImagesPullingTime="2.187584711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:10:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:11:02.819952302 +0000 UTC m=+386.137409344" watchObservedRunningTime="2026-09-11 13:11:02.821671728 +0000 UTC m=+386.139128768"
Sep 11 13:11:03.511677 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:03.511640    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:03.611562 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:03.611528    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:04.994508 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:04.994471    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"]
Sep 11 13:11:05.012215 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.012189    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"]
Sep 11 13:11:05.012397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.012292    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"
Sep 11 13:11:05.015084 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.015056    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:11:05.015194 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.015088    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:11:05.015696 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.015677    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-r6hnf\""
Sep 11 13:11:05.065940 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.065905    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6tjv\" (UniqueName: \"kubernetes.io/projected/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-kube-api-access-x6tjv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct\" (UID: \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"
Sep 11 13:11:05.065940 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.065941    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct\" (UID: \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"
Sep 11 13:11:05.066135 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.065985    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct\" (UID: \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"
Sep 11 13:11:05.166647 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.166612    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6tjv\" (UniqueName: \"kubernetes.io/projected/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-kube-api-access-x6tjv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct\" (UID: \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"
Sep 11 13:11:05.166647 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.166648    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct\" (UID: \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"
Sep 11 13:11:05.166852 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.166673    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct\" (UID: \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"
Sep 11 13:11:05.167041 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.167021    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct\" (UID: \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"
Sep 11 13:11:05.167111 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.167090    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct\" (UID: \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"
Sep 11 13:11:05.175629 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.175606    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6tjv\" (UniqueName: \"kubernetes.io/projected/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-kube-api-access-x6tjv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct\" (UID: \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"
Sep 11 13:11:05.324075 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.324047    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"
Sep 11 13:11:05.613707 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.613680    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"]
Sep 11 13:11:05.616238 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:11:05.616210    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94f6a5e1_9e81_4057_8b57_d27ec9f7ea64.slice/crio-92bdc0a90004b88fb0a03c868010a8f9922fc489649d8ccf0edca365ec1e6abb WatchSource:0}: Error finding container 92bdc0a90004b88fb0a03c868010a8f9922fc489649d8ccf0edca365ec1e6abb: Status 404 returned error can't find the container with id 92bdc0a90004b88fb0a03c868010a8f9922fc489649d8ccf0edca365ec1e6abb
Sep 11 13:11:05.673813 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.673779    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:05.816516 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.816482    2554 generic.go:356] "Generic (PLEG): container finished" podID="94f6a5e1-9e81-4057-8b57-d27ec9f7ea64" containerID="20c3cf48f993e166d6a363342171fc265eb1781b017e15cb31edf983b1fc15cd" exitCode=0
Sep 11 13:11:05.816676 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.816541    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct" event={"ID":"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64","Type":"ContainerDied","Data":"20c3cf48f993e166d6a363342171fc265eb1781b017e15cb31edf983b1fc15cd"}
Sep 11 13:11:05.816676 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:05.816594    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct" event={"ID":"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64","Type":"ContainerStarted","Data":"92bdc0a90004b88fb0a03c868010a8f9922fc489649d8ccf0edca365ec1e6abb"}
Sep 11 13:11:06.822986 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:06.822956    2554 generic.go:356] "Generic (PLEG): container finished" podID="94f6a5e1-9e81-4057-8b57-d27ec9f7ea64" containerID="0f689f7e860901b07127a6e498d5f4aee95cf7e8ca33c35958107f31a40ec580" exitCode=0
Sep 11 13:11:06.823424 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:06.823038    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct" event={"ID":"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64","Type":"ContainerDied","Data":"0f689f7e860901b07127a6e498d5f4aee95cf7e8ca33c35958107f31a40ec580"}
Sep 11 13:11:07.830158 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:07.830127    2554 generic.go:356] "Generic (PLEG): container finished" podID="94f6a5e1-9e81-4057-8b57-d27ec9f7ea64" containerID="11842ee240c72e5839f1b5db13490d2da1f88b64f51e71857df8ead82cdda9b0" exitCode=0
Sep 11 13:11:07.830557 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:07.830211    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct" event={"ID":"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64","Type":"ContainerDied","Data":"11842ee240c72e5839f1b5db13490d2da1f88b64f51e71857df8ead82cdda9b0"}
Sep 11 13:11:08.961599 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:08.961565    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"
Sep 11 13:11:09.098192 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:09.098101    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6tjv\" (UniqueName: \"kubernetes.io/projected/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-kube-api-access-x6tjv\") pod \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\" (UID: \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\") "
Sep 11 13:11:09.098192 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:09.098143    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-util\") pod \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\" (UID: \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\") "
Sep 11 13:11:09.098192 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:09.098196    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-bundle\") pod \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\" (UID: \"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64\") "
Sep 11 13:11:09.099006 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:09.098974    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-bundle" (OuterVolumeSpecName: "bundle") pod "94f6a5e1-9e81-4057-8b57-d27ec9f7ea64" (UID: "94f6a5e1-9e81-4057-8b57-d27ec9f7ea64"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:11:09.100290 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:09.100261    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-kube-api-access-x6tjv" (OuterVolumeSpecName: "kube-api-access-x6tjv") pod "94f6a5e1-9e81-4057-8b57-d27ec9f7ea64" (UID: "94f6a5e1-9e81-4057-8b57-d27ec9f7ea64"). InnerVolumeSpecName "kube-api-access-x6tjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:11:09.103554 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:09.103532    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-util" (OuterVolumeSpecName: "util") pod "94f6a5e1-9e81-4057-8b57-d27ec9f7ea64" (UID: "94f6a5e1-9e81-4057-8b57-d27ec9f7ea64"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:11:09.199611 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:09.199577    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6tjv\" (UniqueName: \"kubernetes.io/projected/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-kube-api-access-x6tjv\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:11:09.199611 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:09.199608    2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-util\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:11:09.199611 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:09.199617    2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94f6a5e1-9e81-4057-8b57-d27ec9f7ea64-bundle\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:11:09.840893 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:09.840856    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct" event={"ID":"94f6a5e1-9e81-4057-8b57-d27ec9f7ea64","Type":"ContainerDied","Data":"92bdc0a90004b88fb0a03c868010a8f9922fc489649d8ccf0edca365ec1e6abb"}
Sep 11 13:11:09.841062 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:09.840929    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="92bdc0a90004b88fb0a03c868010a8f9922fc489649d8ccf0edca365ec1e6abb"
Sep 11 13:11:09.841062 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:09.840872    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg55ct"
Sep 11 13:11:12.267277 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:12.267239    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:12.824478 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:12.824447    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:12.924304 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:12.924271    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:18.115240 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:18.115206    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:18.301243 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:18.301208    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:18.774249 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:18.774211    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:19.069980 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.069918    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:19.197650 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.197616    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:19.241986 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.241953    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"]
Sep 11 13:11:19.242253 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.242242    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94f6a5e1-9e81-4057-8b57-d27ec9f7ea64" containerName="util"
Sep 11 13:11:19.242295 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.242255    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="94f6a5e1-9e81-4057-8b57-d27ec9f7ea64" containerName="util"
Sep 11 13:11:19.242295 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.242275    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94f6a5e1-9e81-4057-8b57-d27ec9f7ea64" containerName="extract"
Sep 11 13:11:19.242295 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.242282    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="94f6a5e1-9e81-4057-8b57-d27ec9f7ea64" containerName="extract"
Sep 11 13:11:19.242295 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.242292    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94f6a5e1-9e81-4057-8b57-d27ec9f7ea64" containerName="pull"
Sep 11 13:11:19.242295 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.242296    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="94f6a5e1-9e81-4057-8b57-d27ec9f7ea64" containerName="pull"
Sep 11 13:11:19.242485 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.242351    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="94f6a5e1-9e81-4057-8b57-d27ec9f7ea64" containerName="extract"
Sep 11 13:11:19.244950 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.244927    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.247198 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.247180    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 13:11:19.247326 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.247251    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 13:11:19.252257 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.252228    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 13:11:19.252388 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.252240    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 13:11:19.252388 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.252265    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-46rbt\""
Sep 11 13:11:19.267769 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.267734    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"]
Sep 11 13:11:19.334493 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.334410    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"]
Sep 11 13:11:19.336709 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.336691    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"
Sep 11 13:11:19.343053 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.342861    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:11:19.343053 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.342864    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:11:19.346170 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.346151    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-r6hnf\""
Sep 11 13:11:19.362130 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.362098    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"]
Sep 11 13:11:19.382005 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.381967    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aa961889-771b-4937-a1fb-d59980ee0b1e-manifests\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.382173 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.382011    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aa961889-771b-4937-a1fb-d59980ee0b1e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.382173 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.382048    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkbxh\" (UniqueName: \"kubernetes.io/projected/aa961889-771b-4937-a1fb-d59980ee0b1e-kube-api-access-mkbxh\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.382173 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.382102    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa961889-771b-4937-a1fb-d59980ee0b1e-tmp\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.382173 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.382158    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa961889-771b-4937-a1fb-d59980ee0b1e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.478572 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.478533    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:19.482634 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.482611    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa961889-771b-4937-a1fb-d59980ee0b1e-tmp\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.482754 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.482647    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa961889-771b-4937-a1fb-d59980ee0b1e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.482754 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.482671    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9vc8\" (UniqueName: \"kubernetes.io/projected/34839758-fbbc-48c4-895b-21789b4941f1-kube-api-access-p9vc8\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2\" (UID: \"34839758-fbbc-48c4-895b-21789b4941f1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"
Sep 11 13:11:19.482754 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.482703    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aa961889-771b-4937-a1fb-d59980ee0b1e-manifests\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.482754 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.482720    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34839758-fbbc-48c4-895b-21789b4941f1-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2\" (UID: \"34839758-fbbc-48c4-895b-21789b4941f1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"
Sep 11 13:11:19.482754 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.482739    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aa961889-771b-4937-a1fb-d59980ee0b1e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.483021 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.482764    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkbxh\" (UniqueName: \"kubernetes.io/projected/aa961889-771b-4937-a1fb-d59980ee0b1e-kube-api-access-mkbxh\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.483021 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.482783    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34839758-fbbc-48c4-895b-21789b4941f1-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2\" (UID: \"34839758-fbbc-48c4-895b-21789b4941f1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"
Sep 11 13:11:19.483136 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.483092    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aa961889-771b-4937-a1fb-d59980ee0b1e-manifests\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.485038 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.485014    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa961889-771b-4937-a1fb-d59980ee0b1e-tmp\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.485144 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.485088    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aa961889-771b-4937-a1fb-d59980ee0b1e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.485144 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.485102    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa961889-771b-4937-a1fb-d59980ee0b1e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.495308 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.495283    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkbxh\" (UniqueName: \"kubernetes.io/projected/aa961889-771b-4937-a1fb-d59980ee0b1e-kube-api-access-mkbxh\") pod \"opendatahub-operator-controller-manager-5c4b86887d-fjpcd\" (UID: \"aa961889-771b-4937-a1fb-d59980ee0b1e\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.556107 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.556055    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:19.576917 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.576884    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:19.584078 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.584048    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34839758-fbbc-48c4-895b-21789b4941f1-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2\" (UID: \"34839758-fbbc-48c4-895b-21789b4941f1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"
Sep 11 13:11:19.584202 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.584112    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9vc8\" (UniqueName: \"kubernetes.io/projected/34839758-fbbc-48c4-895b-21789b4941f1-kube-api-access-p9vc8\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2\" (UID: \"34839758-fbbc-48c4-895b-21789b4941f1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"
Sep 11 13:11:19.584202 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.584154    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34839758-fbbc-48c4-895b-21789b4941f1-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2\" (UID: \"34839758-fbbc-48c4-895b-21789b4941f1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"
Sep 11 13:11:19.584535 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.584492    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34839758-fbbc-48c4-895b-21789b4941f1-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2\" (UID: \"34839758-fbbc-48c4-895b-21789b4941f1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"
Sep 11 13:11:19.584535 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.584506    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34839758-fbbc-48c4-895b-21789b4941f1-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2\" (UID: \"34839758-fbbc-48c4-895b-21789b4941f1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"
Sep 11 13:11:19.597032 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.596994    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9vc8\" (UniqueName: \"kubernetes.io/projected/34839758-fbbc-48c4-895b-21789b4941f1-kube-api-access-p9vc8\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2\" (UID: \"34839758-fbbc-48c4-895b-21789b4941f1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"
Sep 11 13:11:19.646156 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.646125    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"
Sep 11 13:11:19.678945 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.678907    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:19.837386 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.837281    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:19.860018 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.859995    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"]
Sep 11 13:11:19.862321 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:11:19.862293    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa961889_771b_4937_a1fb_d59980ee0b1e.slice/crio-882f77e27af0099c8ea9d796b68f7ec1a6cfc94ada127cbbec35d3064f744ca8 WatchSource:0}: Error finding container 882f77e27af0099c8ea9d796b68f7ec1a6cfc94ada127cbbec35d3064f744ca8: Status 404 returned error can't find the container with id 882f77e27af0099c8ea9d796b68f7ec1a6cfc94ada127cbbec35d3064f744ca8
Sep 11 13:11:19.891175 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.891139    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd" event={"ID":"aa961889-771b-4937-a1fb-d59980ee0b1e","Type":"ContainerStarted","Data":"882f77e27af0099c8ea9d796b68f7ec1a6cfc94ada127cbbec35d3064f744ca8"}
Sep 11 13:11:19.935600 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:19.935568    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"]
Sep 11 13:11:19.938341 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:11:19.938311    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34839758_fbbc_48c4_895b_21789b4941f1.slice/crio-37d0e597886eb5399a1473af0a9c9250ac8295f48a5008a8cb6e534ec141efde WatchSource:0}: Error finding container 37d0e597886eb5399a1473af0a9c9250ac8295f48a5008a8cb6e534ec141efde: Status 404 returned error can't find the container with id 37d0e597886eb5399a1473af0a9c9250ac8295f48a5008a8cb6e534ec141efde
Sep 11 13:11:20.044709 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:20.044666    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:20.149627 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:20.149543    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:20.247400 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:20.247349    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:20.899118 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:20.899080    2554 generic.go:356] "Generic (PLEG): container finished" podID="34839758-fbbc-48c4-895b-21789b4941f1" containerID="b5089fcace30c44decf629b0e733a605b00bf67f1a2dfae03bf9122ab0141745" exitCode=0
Sep 11 13:11:20.899270 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:20.899127    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2" event={"ID":"34839758-fbbc-48c4-895b-21789b4941f1","Type":"ContainerDied","Data":"b5089fcace30c44decf629b0e733a605b00bf67f1a2dfae03bf9122ab0141745"}
Sep 11 13:11:20.899270 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:20.899171    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2" event={"ID":"34839758-fbbc-48c4-895b-21789b4941f1","Type":"ContainerStarted","Data":"37d0e597886eb5399a1473af0a9c9250ac8295f48a5008a8cb6e534ec141efde"}
Sep 11 13:11:23.916904 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:23.916868    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd" event={"ID":"aa961889-771b-4937-a1fb-d59980ee0b1e","Type":"ContainerStarted","Data":"056dff32369a3c018e677b9fc6f7a0b6dff0902938ded5cac98e66b486822d5d"}
Sep 11 13:11:23.918719 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:23.918696    2554 generic.go:356] "Generic (PLEG): container finished" podID="34839758-fbbc-48c4-895b-21789b4941f1" containerID="a113b07dcb9055036f0f70b790bed08d22e847ea90270c14765384b9d95728a1" exitCode=0
Sep 11 13:11:23.918786 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:23.918753    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2" event={"ID":"34839758-fbbc-48c4-895b-21789b4941f1","Type":"ContainerDied","Data":"a113b07dcb9055036f0f70b790bed08d22e847ea90270c14765384b9d95728a1"}
Sep 11 13:11:24.926467 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:24.926434    2554 generic.go:356] "Generic (PLEG): container finished" podID="34839758-fbbc-48c4-895b-21789b4941f1" containerID="d709e44eee7c0a0228f3cb395882b6e4ddbedcc3f1e9673744004d346c9d520b" exitCode=0
Sep 11 13:11:24.926869 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:24.926522    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2" event={"ID":"34839758-fbbc-48c4-895b-21789b4941f1","Type":"ContainerDied","Data":"d709e44eee7c0a0228f3cb395882b6e4ddbedcc3f1e9673744004d346c9d520b"}
Sep 11 13:11:24.928003 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:24.927977    2554 generic.go:356] "Generic (PLEG): container finished" podID="aa961889-771b-4937-a1fb-d59980ee0b1e" containerID="056dff32369a3c018e677b9fc6f7a0b6dff0902938ded5cac98e66b486822d5d" exitCode=0
Sep 11 13:11:24.928104 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:24.928022    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd" event={"ID":"aa961889-771b-4937-a1fb-d59980ee0b1e","Type":"ContainerDied","Data":"056dff32369a3c018e677b9fc6f7a0b6dff0902938ded5cac98e66b486822d5d"}
Sep 11 13:11:25.935591 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:25.935533    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd" event={"ID":"aa961889-771b-4937-a1fb-d59980ee0b1e","Type":"ContainerStarted","Data":"2adc163c17155a7dccbdd2353b4a5462f56c8f1328f3449473323c85f2c2574f"}
Sep 11 13:11:25.936068 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:25.935654    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:25.948203 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:25.947670    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:25.971599 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:25.971482    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd" podStartSLOduration=2.351164199 podStartE2EDuration="6.971465489s" totalImagesPullingTime="3.55098389s" totalInitContainerRuntime="1.0693174s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:11:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:11:25.970155078 +0000 UTC m=+409.287612119" watchObservedRunningTime="2026-09-11 13:11:25.971465489 +0000 UTC m=+409.288922528"
Sep 11 13:11:26.065132 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.065107    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:26.065742 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.065724    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"
Sep 11 13:11:26.140313 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.140277    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34839758-fbbc-48c4-895b-21789b4941f1-bundle\") pod \"34839758-fbbc-48c4-895b-21789b4941f1\" (UID: \"34839758-fbbc-48c4-895b-21789b4941f1\") "
Sep 11 13:11:26.140313 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.140320    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34839758-fbbc-48c4-895b-21789b4941f1-util\") pod \"34839758-fbbc-48c4-895b-21789b4941f1\" (UID: \"34839758-fbbc-48c4-895b-21789b4941f1\") "
Sep 11 13:11:26.140534 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.140417    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9vc8\" (UniqueName: \"kubernetes.io/projected/34839758-fbbc-48c4-895b-21789b4941f1-kube-api-access-p9vc8\") pod \"34839758-fbbc-48c4-895b-21789b4941f1\" (UID: \"34839758-fbbc-48c4-895b-21789b4941f1\") "
Sep 11 13:11:26.141129 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.141101    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34839758-fbbc-48c4-895b-21789b4941f1-bundle" (OuterVolumeSpecName: "bundle") pod "34839758-fbbc-48c4-895b-21789b4941f1" (UID: "34839758-fbbc-48c4-895b-21789b4941f1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:11:26.142426 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.142397    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34839758-fbbc-48c4-895b-21789b4941f1-kube-api-access-p9vc8" (OuterVolumeSpecName: "kube-api-access-p9vc8") pod "34839758-fbbc-48c4-895b-21789b4941f1" (UID: "34839758-fbbc-48c4-895b-21789b4941f1"). InnerVolumeSpecName "kube-api-access-p9vc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:11:26.147349 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.147323    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34839758-fbbc-48c4-895b-21789b4941f1-util" (OuterVolumeSpecName: "util") pod "34839758-fbbc-48c4-895b-21789b4941f1" (UID: "34839758-fbbc-48c4-895b-21789b4941f1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:11:26.241792 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.241707    2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/34839758-fbbc-48c4-895b-21789b4941f1-bundle\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:11:26.241792 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.241736    2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/34839758-fbbc-48c4-895b-21789b4941f1-util\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:11:26.241792 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.241745    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9vc8\" (UniqueName: \"kubernetes.io/projected/34839758-fbbc-48c4-895b-21789b4941f1-kube-api-access-p9vc8\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:11:26.942316 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.942287    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2"
Sep 11 13:11:26.942316 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.942295    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02jfh2" event={"ID":"34839758-fbbc-48c4-895b-21789b4941f1","Type":"ContainerDied","Data":"37d0e597886eb5399a1473af0a9c9250ac8295f48a5008a8cb6e534ec141efde"}
Sep 11 13:11:26.942864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:26.942341    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37d0e597886eb5399a1473af0a9c9250ac8295f48a5008a8cb6e534ec141efde"
Sep 11 13:11:31.215147 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:31.215114    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:31.309673 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:31.309636    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:36.944528 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:36.944497    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-fjpcd"
Sep 11 13:11:38.646220 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:38.646175    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:38.853898 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:38.853865    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:49.543015 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.542975    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:49.594335 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.594303    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"]
Sep 11 13:11:49.594658 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.594645    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34839758-fbbc-48c4-895b-21789b4941f1" containerName="pull"
Sep 11 13:11:49.594700 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.594660    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="34839758-fbbc-48c4-895b-21789b4941f1" containerName="pull"
Sep 11 13:11:49.594700 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.594667    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34839758-fbbc-48c4-895b-21789b4941f1" containerName="extract"
Sep 11 13:11:49.594700 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.594673    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="34839758-fbbc-48c4-895b-21789b4941f1" containerName="extract"
Sep 11 13:11:49.594700 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.594679    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34839758-fbbc-48c4-895b-21789b4941f1" containerName="util"
Sep 11 13:11:49.594700 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.594684    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="34839758-fbbc-48c4-895b-21789b4941f1" containerName="util"
Sep 11 13:11:49.594859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.594741    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="34839758-fbbc-48c4-895b-21789b4941f1" containerName="extract"
Sep 11 13:11:49.601505 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.601477    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"
Sep 11 13:11:49.603877 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.603850    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 13:11:49.603877 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.603856    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 13:11:49.611300 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.611272    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"]
Sep 11 13:11:49.630851 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.630822    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e27e323-a30b-4bc3-8b06-437d4ada1fb9-tmp\") pod \"kube-auth-proxy-74db4b4cbd-b5qwr\" (UID: \"4e27e323-a30b-4bc3-8b06-437d4ada1fb9\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"
Sep 11 13:11:49.630992 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.630909    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e27e323-a30b-4bc3-8b06-437d4ada1fb9-tls-certs\") pod \"kube-auth-proxy-74db4b4cbd-b5qwr\" (UID: \"4e27e323-a30b-4bc3-8b06-437d4ada1fb9\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"
Sep 11 13:11:49.630992 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.630948    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jh5c\" (UniqueName: \"kubernetes.io/projected/4e27e323-a30b-4bc3-8b06-437d4ada1fb9-kube-api-access-9jh5c\") pod \"kube-auth-proxy-74db4b4cbd-b5qwr\" (UID: \"4e27e323-a30b-4bc3-8b06-437d4ada1fb9\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"
Sep 11 13:11:49.657073 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.657036    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:49.732085 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.732051    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jh5c\" (UniqueName: \"kubernetes.io/projected/4e27e323-a30b-4bc3-8b06-437d4ada1fb9-kube-api-access-9jh5c\") pod \"kube-auth-proxy-74db4b4cbd-b5qwr\" (UID: \"4e27e323-a30b-4bc3-8b06-437d4ada1fb9\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"
Sep 11 13:11:49.732283 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.732103    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e27e323-a30b-4bc3-8b06-437d4ada1fb9-tmp\") pod \"kube-auth-proxy-74db4b4cbd-b5qwr\" (UID: \"4e27e323-a30b-4bc3-8b06-437d4ada1fb9\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"
Sep 11 13:11:49.732283 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.732138    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e27e323-a30b-4bc3-8b06-437d4ada1fb9-tls-certs\") pod \"kube-auth-proxy-74db4b4cbd-b5qwr\" (UID: \"4e27e323-a30b-4bc3-8b06-437d4ada1fb9\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"
Sep 11 13:11:49.734490 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.734458    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e27e323-a30b-4bc3-8b06-437d4ada1fb9-tmp\") pod \"kube-auth-proxy-74db4b4cbd-b5qwr\" (UID: \"4e27e323-a30b-4bc3-8b06-437d4ada1fb9\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"
Sep 11 13:11:49.734638 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.734622    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e27e323-a30b-4bc3-8b06-437d4ada1fb9-tls-certs\") pod \"kube-auth-proxy-74db4b4cbd-b5qwr\" (UID: \"4e27e323-a30b-4bc3-8b06-437d4ada1fb9\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"
Sep 11 13:11:49.739637 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.739619    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jh5c\" (UniqueName: \"kubernetes.io/projected/4e27e323-a30b-4bc3-8b06-437d4ada1fb9-kube-api-access-9jh5c\") pod \"kube-auth-proxy-74db4b4cbd-b5qwr\" (UID: \"4e27e323-a30b-4bc3-8b06-437d4ada1fb9\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"
Sep 11 13:11:49.751583 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.751559    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:49.853311 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.853278    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:49.911180 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:49.911145    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"
Sep 11 13:11:50.116398 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.116294    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:50.128194 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.128160    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-574z2"]
Sep 11 13:11:50.130665 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.130645    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-574z2"
Sep 11 13:11:50.133006 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.132985    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 11 13:11:50.133112 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.133010    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-b29ms\""
Sep 11 13:11:50.134043 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.134017    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fd996fe-1e9a-4a82-a37a-d1161f54177b-tls-certs\") pod \"model-serving-api-54d89bd5fc-574z2\" (UID: \"1fd996fe-1e9a-4a82-a37a-d1161f54177b\") " pod="opendatahub/model-serving-api-54d89bd5fc-574z2"
Sep 11 13:11:50.134149 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.134068    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhhth\" (UniqueName: \"kubernetes.io/projected/1fd996fe-1e9a-4a82-a37a-d1161f54177b-kube-api-access-rhhth\") pod \"model-serving-api-54d89bd5fc-574z2\" (UID: \"1fd996fe-1e9a-4a82-a37a-d1161f54177b\") " pod="opendatahub/model-serving-api-54d89bd5fc-574z2"
Sep 11 13:11:50.143113 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.143090    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-574z2"]
Sep 11 13:11:50.198662 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.198634    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr"]
Sep 11 13:11:50.200970 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:11:50.200940    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e27e323_a30b_4bc3_8b06_437d4ada1fb9.slice/crio-4f15a1cfccd9485cbada34e3371a633cdf619a1b0beb795d85bac35285b39a7f WatchSource:0}: Error finding container 4f15a1cfccd9485cbada34e3371a633cdf619a1b0beb795d85bac35285b39a7f: Status 404 returned error can't find the container with id 4f15a1cfccd9485cbada34e3371a633cdf619a1b0beb795d85bac35285b39a7f
Sep 11 13:11:50.235196 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.235159    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fd996fe-1e9a-4a82-a37a-d1161f54177b-tls-certs\") pod \"model-serving-api-54d89bd5fc-574z2\" (UID: \"1fd996fe-1e9a-4a82-a37a-d1161f54177b\") " pod="opendatahub/model-serving-api-54d89bd5fc-574z2"
Sep 11 13:11:50.235398 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.235210    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhhth\" (UniqueName: \"kubernetes.io/projected/1fd996fe-1e9a-4a82-a37a-d1161f54177b-kube-api-access-rhhth\") pod \"model-serving-api-54d89bd5fc-574z2\" (UID: \"1fd996fe-1e9a-4a82-a37a-d1161f54177b\") " pod="opendatahub/model-serving-api-54d89bd5fc-574z2"
Sep 11 13:11:50.235398 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:11:50.235319    2554 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 11 13:11:50.235479 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:11:50.235418    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fd996fe-1e9a-4a82-a37a-d1161f54177b-tls-certs podName:1fd996fe-1e9a-4a82-a37a-d1161f54177b nodeName:}" failed. No retries permitted until 2026-09-11 13:11:50.735399485 +0000 UTC m=+434.052856523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/1fd996fe-1e9a-4a82-a37a-d1161f54177b-tls-certs") pod "model-serving-api-54d89bd5fc-574z2" (UID: "1fd996fe-1e9a-4a82-a37a-d1161f54177b") : secret "model-serving-api-tls" not found
Sep 11 13:11:50.244256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.244231    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhhth\" (UniqueName: \"kubernetes.io/projected/1fd996fe-1e9a-4a82-a37a-d1161f54177b-kube-api-access-rhhth\") pod \"model-serving-api-54d89bd5fc-574z2\" (UID: \"1fd996fe-1e9a-4a82-a37a-d1161f54177b\") " pod="opendatahub/model-serving-api-54d89bd5fc-574z2"
Sep 11 13:11:50.662612 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.662574    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:50.739310 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.739264    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fd996fe-1e9a-4a82-a37a-d1161f54177b-tls-certs\") pod \"model-serving-api-54d89bd5fc-574z2\" (UID: \"1fd996fe-1e9a-4a82-a37a-d1161f54177b\") " pod="opendatahub/model-serving-api-54d89bd5fc-574z2"
Sep 11 13:11:50.742169 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:50.742145    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fd996fe-1e9a-4a82-a37a-d1161f54177b-tls-certs\") pod \"model-serving-api-54d89bd5fc-574z2\" (UID: \"1fd996fe-1e9a-4a82-a37a-d1161f54177b\") " pod="opendatahub/model-serving-api-54d89bd5fc-574z2"
Sep 11 13:11:51.041394 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:51.041081    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-574z2"
Sep 11 13:11:51.046924 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:51.046884    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:51.065464 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:51.065354    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr" event={"ID":"4e27e323-a30b-4bc3-8b06-437d4ada1fb9","Type":"ContainerStarted","Data":"4f15a1cfccd9485cbada34e3371a633cdf619a1b0beb795d85bac35285b39a7f"}
Sep 11 13:11:51.140277 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:51.140190    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:51.369095 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:51.369060    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-574z2"]
Sep 11 13:11:51.372039 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:11:51.372008    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fd996fe_1e9a_4a82_a37a_d1161f54177b.slice/crio-13fdc92d72d71a68e58c43a4fa641a9ab41f8fba624af6656d761593aac63381 WatchSource:0}: Error finding container 13fdc92d72d71a68e58c43a4fa641a9ab41f8fba624af6656d761593aac63381: Status 404 returned error can't find the container with id 13fdc92d72d71a68e58c43a4fa641a9ab41f8fba624af6656d761593aac63381
Sep 11 13:11:52.071935 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:52.071894    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-574z2" event={"ID":"1fd996fe-1e9a-4a82-a37a-d1161f54177b","Type":"ContainerStarted","Data":"13fdc92d72d71a68e58c43a4fa641a9ab41f8fba624af6656d761593aac63381"}
Sep 11 13:11:52.387691 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:52.387602    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:53.078618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:53.078584    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr" event={"ID":"4e27e323-a30b-4bc3-8b06-437d4ada1fb9","Type":"ContainerStarted","Data":"256b274eae6d408707aaa4be394292501b8c95dcf418a92718c09111606415e6"}
Sep 11 13:11:53.119221 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:53.119163    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-b5qwr" podStartSLOduration=1.345043625 podStartE2EDuration="4.119145046s" totalImagesPullingTime="2.774101421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:11:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:11:53.117221553 +0000 UTC m=+436.434678595" watchObservedRunningTime="2026-09-11 13:11:53.119145046 +0000 UTC m=+436.436602086"
Sep 11 13:11:55.090765 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:55.090722    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-574z2" event={"ID":"1fd996fe-1e9a-4a82-a37a-d1161f54177b","Type":"ContainerStarted","Data":"73a45fe739d72a64539c931b6f5a04973051cf90a882cc0d7c3ff8dda557ff71"}
Sep 11 13:11:55.091139 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:55.090878    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-574z2"
Sep 11 13:11:55.110842 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:55.110793    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-574z2" podStartSLOduration=2.306499128 podStartE2EDuration="5.110777845s" totalImagesPullingTime="2.804278717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:11:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:11:55.109197001 +0000 UTC m=+438.426654042" watchObservedRunningTime="2026-09-11 13:11:55.110777845 +0000 UTC m=+438.428234885"
Sep 11 13:11:56.416175 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:56.416139    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:56.515411 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:56.515352    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:56.931745 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:56.931708    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"]
Sep 11 13:11:56.934263 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:56.934239    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:56.938637 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:56.938617    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 13:11:56.938727 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:56.938617    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 13:11:56.938727 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:56.938624    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6nhx5\""
Sep 11 13:11:56.938937 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:56.938922    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 13:11:56.939010 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:56.938977    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 13:11:56.949761 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:56.949737    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"]
Sep 11 13:11:57.002084 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.002052    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.002264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.002095    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.002264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.002119    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.002264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.002141    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5l67\" (UniqueName: \"kubernetes.io/projected/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-kube-api-access-l5l67\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.002264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.002161    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.002264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.002224    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.002264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.002255    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.103226 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.103186    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5l67\" (UniqueName: \"kubernetes.io/projected/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-kube-api-access-l5l67\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.103226 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.103228    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.103513 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.103255    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.103513 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.103278    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.103513 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.103329    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.103513 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.103376    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.103513 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.103411    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.103995 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.103966    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.106391 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.106341    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.106503 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.106459    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.106584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.106514    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.106584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.106563    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.118658 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.118631    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5l67\" (UniqueName: \"kubernetes.io/projected/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-kube-api-access-l5l67\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.129164 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.129135    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b62a2b8-0ec4-4a9d-8816-8350c8a49367-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-4gmk5\" (UID: \"0b62a2b8-0ec4-4a9d-8816-8350c8a49367\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.244236 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.244131    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:11:57.580679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.580648    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"]
Sep 11 13:11:57.584088 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:11:57.584050    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b62a2b8_0ec4_4a9d_8816_8350c8a49367.slice/crio-51e35aca7f1db56f4cfce3da86932a1866c57b80514bfdc45c98fa6f59ff745b WatchSource:0}: Error finding container 51e35aca7f1db56f4cfce3da86932a1866c57b80514bfdc45c98fa6f59ff745b: Status 404 returned error can't find the container with id 51e35aca7f1db56f4cfce3da86932a1866c57b80514bfdc45c98fa6f59ff745b
Sep 11 13:11:57.595739 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.595695    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:57.990204 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:57.990095    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:58.110809 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.110767    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5" event={"ID":"0b62a2b8-0ec4-4a9d-8816-8350c8a49367","Type":"ContainerStarted","Data":"51e35aca7f1db56f4cfce3da86932a1866c57b80514bfdc45c98fa6f59ff745b"}
Sep 11 13:11:58.122279 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.122241    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:11:58.239882 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.239848    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"]
Sep 11 13:11:58.244544 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.244469    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.248668 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.248447    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 13:11:58.254472 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.254262    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 13:11:58.254472 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.254316    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 13:11:58.254844 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.254631    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-q72pw\""
Sep 11 13:11:58.274182 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.274114    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"]
Sep 11 13:11:58.314536 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.314494    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp2pg\" (UniqueName: \"kubernetes.io/projected/8a6d1998-9e69-44db-9662-a8f8d4700f7f-kube-api-access-xp2pg\") pod \"lws-controller-manager-5f8bd79dcc-nz8qv\" (UID: \"8a6d1998-9e69-44db-9662-a8f8d4700f7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.314812 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.314559    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a6d1998-9e69-44db-9662-a8f8d4700f7f-metrics-cert\") pod \"lws-controller-manager-5f8bd79dcc-nz8qv\" (UID: \"8a6d1998-9e69-44db-9662-a8f8d4700f7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.314812 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.314589    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a6d1998-9e69-44db-9662-a8f8d4700f7f-manager-config\") pod \"lws-controller-manager-5f8bd79dcc-nz8qv\" (UID: \"8a6d1998-9e69-44db-9662-a8f8d4700f7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.314812 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.314672    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a6d1998-9e69-44db-9662-a8f8d4700f7f-cert\") pod \"lws-controller-manager-5f8bd79dcc-nz8qv\" (UID: \"8a6d1998-9e69-44db-9662-a8f8d4700f7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.415805 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.415764    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp2pg\" (UniqueName: \"kubernetes.io/projected/8a6d1998-9e69-44db-9662-a8f8d4700f7f-kube-api-access-xp2pg\") pod \"lws-controller-manager-5f8bd79dcc-nz8qv\" (UID: \"8a6d1998-9e69-44db-9662-a8f8d4700f7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.415998 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.415857    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a6d1998-9e69-44db-9662-a8f8d4700f7f-metrics-cert\") pod \"lws-controller-manager-5f8bd79dcc-nz8qv\" (UID: \"8a6d1998-9e69-44db-9662-a8f8d4700f7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.415998 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.415898    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a6d1998-9e69-44db-9662-a8f8d4700f7f-manager-config\") pod \"lws-controller-manager-5f8bd79dcc-nz8qv\" (UID: \"8a6d1998-9e69-44db-9662-a8f8d4700f7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.415998 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.415965    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a6d1998-9e69-44db-9662-a8f8d4700f7f-cert\") pod \"lws-controller-manager-5f8bd79dcc-nz8qv\" (UID: \"8a6d1998-9e69-44db-9662-a8f8d4700f7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.418037 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.417995    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a6d1998-9e69-44db-9662-a8f8d4700f7f-manager-config\") pod \"lws-controller-manager-5f8bd79dcc-nz8qv\" (UID: \"8a6d1998-9e69-44db-9662-a8f8d4700f7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.421815 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.421784    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a6d1998-9e69-44db-9662-a8f8d4700f7f-metrics-cert\") pod \"lws-controller-manager-5f8bd79dcc-nz8qv\" (UID: \"8a6d1998-9e69-44db-9662-a8f8d4700f7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.422293 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.422271    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a6d1998-9e69-44db-9662-a8f8d4700f7f-cert\") pod \"lws-controller-manager-5f8bd79dcc-nz8qv\" (UID: \"8a6d1998-9e69-44db-9662-a8f8d4700f7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.428235 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.428209    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp2pg\" (UniqueName: \"kubernetes.io/projected/8a6d1998-9e69-44db-9662-a8f8d4700f7f-kube-api-access-xp2pg\") pod \"lws-controller-manager-5f8bd79dcc-nz8qv\" (UID: \"8a6d1998-9e69-44db-9662-a8f8d4700f7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.557044 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.556947    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:11:58.846509 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:58.846451    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"]
Sep 11 13:11:58.849675 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:11:58.849641    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6d1998_9e69_44db_9662_a8f8d4700f7f.slice/crio-24709300b9529a6a8c08b9bfbe5295c27ad16c63c1526dd4b06b8580b1a7c9d5 WatchSource:0}: Error finding container 24709300b9529a6a8c08b9bfbe5295c27ad16c63c1526dd4b06b8580b1a7c9d5: Status 404 returned error can't find the container with id 24709300b9529a6a8c08b9bfbe5295c27ad16c63c1526dd4b06b8580b1a7c9d5
Sep 11 13:11:59.118151 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:59.118081    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv" event={"ID":"8a6d1998-9e69-44db-9662-a8f8d4700f7f","Type":"ContainerStarted","Data":"24709300b9529a6a8c08b9bfbe5295c27ad16c63c1526dd4b06b8580b1a7c9d5"}
Sep 11 13:11:59.866168 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:59.866131    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:11:59.866447 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:11:59.866202    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:12:00.127776 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:00.127679    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5" event={"ID":"0b62a2b8-0ec4-4a9d-8816-8350c8a49367","Type":"ContainerStarted","Data":"c8e45efe3a2060f99bb6148f266473837da55a13c2a316ad5be10525c53f7584"}
Sep 11 13:12:00.127776 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:00.127748    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:12:00.129626 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:00.129591    2554 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-4gmk5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 13:12:00.129761 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:00.129648    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5" podUID="0b62a2b8-0ec4-4a9d-8816-8350c8a49367" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:12:01.135824 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:01.135791    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv" event={"ID":"8a6d1998-9e69-44db-9662-a8f8d4700f7f","Type":"ContainerStarted","Data":"34fce93cea94e7cc6b698aecefafb3d98255c61f80a4a1d8e09f331cd80c2fc0"}
Sep 11 13:12:01.136300 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:01.136048    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:12:01.136516 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:01.136490    2554 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-4gmk5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 13:12:01.136646 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:01.136541    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5" podUID="0b62a2b8-0ec4-4a9d-8816-8350c8a49367" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:12:01.209936 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:01.209879    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv" podStartSLOduration=1.271113024 podStartE2EDuration="3.209858126s" totalImagesPullingTime="1.938745102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:12:01.209709521 +0000 UTC m=+444.527166563" watchObservedRunningTime="2026-09-11 13:12:01.209858126 +0000 UTC m=+444.527315184"
Sep 11 13:12:01.210682 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:01.210638    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5" podStartSLOduration=2.931189051 podStartE2EDuration="5.21062578s" totalImagesPullingTime="2.279436729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:12:00.227488961 +0000 UTC m=+443.544946001" watchObservedRunningTime="2026-09-11 13:12:01.21062578 +0000 UTC m=+444.528082837"
Sep 11 13:12:01.264469 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:01.264424    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:12:01.506654 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:01.506531    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:12:01.680938 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:01.680910    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:12:01.950753 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:01.950720    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:12:02.210385 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:02.210345    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:12:04.137552 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:04.137514    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gmk5"
Sep 11 13:12:06.100193 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:06.100163    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-574z2"
Sep 11 13:12:12.144113 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:12.144082    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-nz8qv"
Sep 11 13:12:50.720082 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:50.720036    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:12:50.820496 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:12:50.820453    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:15.592293 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:15.592259    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:29.000652 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:29.000606    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:31.811229 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:31.811123    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:32.230074 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:32.230037    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:32.521854 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:32.521752    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:34.601942 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:34.601905    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:35.309222 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:35.309185    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:35.684899 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:35.684860    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:39.198559 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:39.198511    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:39.830267 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:39.830234    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:40.304318 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:40.304206    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:40.728979 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:40.728950    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:45.440129 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:45.440092    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:45.628875 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:45.628833    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:50.065004 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:50.064961    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:50.161422 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:50.161388    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:50.691131 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:50.691095    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:53.018308 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:53.018249    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:53.186487 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:53.186453    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:53.372405 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:53.372344    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:58.027621 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:58.027580    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:58.186825 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:58.186792    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:13:58.317329 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:13:58.317299    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:05.282625 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:05.282594    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:05.663431 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:05.663392    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:10.289595 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:10.289559    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:10.453688 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:10.453654    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:10.669456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:10.669418    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:10.948926 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:10.948833    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:11.356269 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:11.356237    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:12.970202 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:12.970163    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:13.144940 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:13.144908    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:18.070488 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:18.070453    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:18.280682 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:18.280649    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:18.480820 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:18.480729    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:36.996523 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:36.996486    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:37.090777 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:37.090750    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:14:37.093061 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:37.093035    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:14:37.097558 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:37.097530    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:41.869301 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:41.869263    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:42.171629 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:42.171544    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:46.584818 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:46.584776    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:14:48.048155 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:14:48.048120    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:02.736588 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:02.736555    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:02.840751 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:02.840708    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:03.157657 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:03.157615    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:12.892860 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:12.892818    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:13.021343 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:13.021308    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:19.480532 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:19.480491    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:19.554471 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:19.554396    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:19.613321 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:19.613285    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:19.802457 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:19.802323    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:19.903215 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:19.903185    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:22.875606 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:22.875571    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:22.981820 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:22.981785    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:45.335661 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:45.335631    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:45.409966 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:45.409928    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:48.309885 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:48.309850    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:48.375516 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:48.375486    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:54.629653 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:54.629613    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:54.726153 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:54.726114    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:56.044444 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:56.044409    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:56.145202 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:56.145158    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:56.270995 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:56.270955    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:56.624414 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:56.624337    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:56.988338 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:56.988249    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:15:57.108515 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:15:57.108470    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:16:08.018708 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:16:08.018670    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:16:08.151972 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:16:08.151932    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:16:29.446890 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:16:29.446855    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:16:34.176717 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:16:34.176677    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:16:34.274834 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:16:34.274795    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:16:40.686079 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:16:40.686035    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:17:02.368500 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:17:02.368466    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:17:08.924536 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:17:08.924473    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:17:18.727803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:17:18.727765    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:17:18.836602 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:17:18.836566    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:17:20.709878 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:17:20.709848    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:17:31.972724 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:17:31.972687    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:17:32.074744 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:17:32.074707    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:17:38.475070 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:17:38.475030    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:17:39.621787 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:17:39.621756    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:17:44.698779 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:17:44.698748    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:03.230550 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:03.230515    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:21.440725 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:21.440664    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:21.541434 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:21.541399    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:22.529800 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:22.529763    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:22.638428 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:22.638392    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:37.123862 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:37.123837    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:19:37.124449 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:37.123841    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:19:38.398208 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:38.398171    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:38.517115 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:38.517078    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:38.777558 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:38.777477    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:39.147102 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:39.147066    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:39.907101 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:39.907065    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:40.009235 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:40.009190    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:40.376684 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:40.376647    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:42.018593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:42.018559    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:42.114413 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:42.114350    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:43.266614 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:43.266586    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:43.378541 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:43.378506    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:43.473471 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:43.473438    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:43.950724 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:43.950694    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:44.060070 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:44.060021    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:45.407975 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:45.407931    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:45.638386 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:45.638324    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:46.354534 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:46.354496    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:47.307720 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:47.307690    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:47.430225 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:47.430191    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:47.524023 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:47.523979    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:47.854286 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:47.854253    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:48.096352 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:48.096315    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:49.511354 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:49.511320    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:49.753745 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:49.753716    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:50.456894 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:50.456858    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:50.553875 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:50.553841    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:50.799128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:50.799040    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:51.000595 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:51.000565    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:51.114471 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:51.114441    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:51.517061 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:51.516972    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:51.621505 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:51.621469    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:52.129310 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.129279    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"]
Sep 11 13:19:52.131557 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.131539    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:52.136489 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.136466    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 13:19:52.136627 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.136544    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-687dcb39-serving-cert\""
Sep 11 13:19:52.137045 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.137015    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:19:52.137191 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.137168    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 13:19:52.137299 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.137274    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zxpw2\""
Sep 11 13:19:52.148202 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.148178    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"]
Sep 11 13:19:52.160536 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.160501    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-maas-api-tls\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:52.160672 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.160548    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-openshift-service-ca\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:52.160672 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.160606    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dhxg\" (UniqueName: \"kubernetes.io/projected/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-kube-api-access-5dhxg\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:52.160672 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.160625    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-metrics-certs\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:52.192571 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.192542    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:52.262051 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.262003    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-metrics-certs\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:52.262275 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.262105    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-maas-api-tls\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:52.262275 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.262135    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-openshift-service-ca\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:52.262275 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.262199    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dhxg\" (UniqueName: \"kubernetes.io/projected/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-kube-api-access-5dhxg\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:52.262275 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:19:52.262256    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-serving-cert: secret "maas-api-e2e-ait-687dcb39-serving-cert" not found
Sep 11 13:19:52.262522 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:19:52.262330    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-maas-api-tls podName:aadc4c8b-b824-4433-bec7-ab18bc8c2ab2 nodeName:}" failed. No retries permitted until 2026-09-11 13:19:52.762310489 +0000 UTC m=+916.079767512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-maas-api-tls") pod "maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd" (UID: "aadc4c8b-b824-4433-bec7-ab18bc8c2ab2") : secret "maas-api-e2e-ait-687dcb39-serving-cert" not found
Sep 11 13:19:52.262944 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.262921    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-openshift-service-ca\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:52.264847 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.264821    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-metrics-certs\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:52.274016 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.273986    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dhxg\" (UniqueName: \"kubernetes.io/projected/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-kube-api-access-5dhxg\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:52.292088 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.292066    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:52.750608 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.750575    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:52.766515 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.766486    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-maas-api-tls\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:52.766677 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:19:52.766625    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-serving-cert: secret "maas-api-e2e-ait-687dcb39-serving-cert" not found
Sep 11 13:19:52.766732 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:19:52.766704    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-maas-api-tls podName:aadc4c8b-b824-4433-bec7-ab18bc8c2ab2 nodeName:}" failed. No retries permitted until 2026-09-11 13:19:53.766688306 +0000 UTC m=+917.084145324 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-maas-api-tls") pod "maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd" (UID: "aadc4c8b-b824-4433-bec7-ab18bc8c2ab2") : secret "maas-api-e2e-ait-687dcb39-serving-cert" not found
Sep 11 13:19:52.866337 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.866296    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:52.947216 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:52.947183    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:53.254070 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:53.254037    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:53.775319 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:53.775281    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-maas-api-tls\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:53.777720 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:53.777697    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-maas-api-tls\") pod \"maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:53.942941 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:53.942908    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:54.125381 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:54.125327    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:54.216293 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:54.216269    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"]
Sep 11 13:19:54.218291 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:19:54.218264    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaadc4c8b_b824_4433_bec7_ab18bc8c2ab2.slice/crio-c8e29b17162edca213d32645b84e81aaf72fc4bd6e4bec790262f67a49229272 WatchSource:0}: Error finding container c8e29b17162edca213d32645b84e81aaf72fc4bd6e4bec790262f67a49229272: Status 404 returned error can't find the container with id c8e29b17162edca213d32645b84e81aaf72fc4bd6e4bec790262f67a49229272
Sep 11 13:19:54.219715 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:54.219698    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:19:54.422762 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:54.422680    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd" event={"ID":"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2","Type":"ContainerStarted","Data":"c8e29b17162edca213d32645b84e81aaf72fc4bd6e4bec790262f67a49229272"}
Sep 11 13:19:54.968663 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:54.968626    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:55.744629 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:55.744593    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:56.500687 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:56.500662    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:56.715088 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:56.715007    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:57.112994 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:57.112959    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:57.316938 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:57.316894    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:57.448314 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:57.448181    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd" event={"ID":"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2","Type":"ContainerStarted","Data":"a9add5140314a792419a9bc21f05ca944abd0e7d2b535ae90154f714514354e0"}
Sep 11 13:19:57.448501 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:57.448385    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:19:57.459687 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:57.459661    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:57.468203 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:57.468153    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd" podStartSLOduration=3.254735971 podStartE2EDuration="5.468137448s" totalImagesPullingTime="2.213401477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:57.467312502 +0000 UTC m=+920.784769551" watchObservedRunningTime="2026-09-11 13:19:57.468137448 +0000 UTC m=+920.785594487"
Sep 11 13:19:57.560498 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:57.560461    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:57.690822 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:57.690787    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:57.783120 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:57.783037    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:57.887764 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:57.887726    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:57.984566 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:57.984532    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:58.395497 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:58.395462    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:19:59.461231 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:19:59.461198    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:00.083173 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:00.083142    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:00.183715 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:00.183683    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:00.608820 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:00.608780    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:00.696039 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:00.696003    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:01.778643 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:01.778607    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:02.755599 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:02.755564    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:02.937869 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:02.937835    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:03.458881 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:03.458854    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:20:03.646541 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:03.646510    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:03.794768 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:03.794690    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:03.894304 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:03.894267    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:04.260229 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:04.260193    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:04.357034 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:04.357004    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:04.774727 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:04.774692    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:05.890394 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:05.890345    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:06.807580 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:06.807529    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:07.562070 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:07.562039    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:08.372643 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:08.372612    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:09.279003 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:09.278969    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:10.049895 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:10.049858    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:11.038480 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:11.038445    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:11.917972 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:11.917935    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:12.641326 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:12.641290    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:13.545036 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:13.545008    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:14.473642 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:14.473607    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:14.766669 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:14.766582    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:15.444402 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:15.444359    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:16.279978 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:16.279936    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:17.277410 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:17.277379    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:18.084786 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:18.084744    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:18.423341 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:18.423255    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:18.540021 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:18.539993    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:18.663471 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:18.663436    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:21.620940 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:21.620900    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"]
Sep 11 13:20:21.621388 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:21.621198    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd" podUID="aadc4c8b-b824-4433-bec7-ab18bc8c2ab2" containerName="maas-api" containerID="cri-o://a9add5140314a792419a9bc21f05ca944abd0e7d2b535ae90154f714514354e0" gracePeriod=30
Sep 11 13:20:21.655478 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:21.655441    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:21.740381 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:21.740334    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:21.840843 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:21.840798    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:21.867823 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:21.867799    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:20:21.932004 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:21.931913    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:22.018041 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.018007    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-metrics-certs\") pod \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") "
Sep 11 13:20:22.018245 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.018111    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-maas-api-tls\") pod \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") "
Sep 11 13:20:22.018245 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.018175    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-openshift-service-ca\") pod \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") "
Sep 11 13:20:22.018245 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.018243    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dhxg\" (UniqueName: \"kubernetes.io/projected/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-kube-api-access-5dhxg\") pod \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\" (UID: \"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2\") "
Sep 11 13:20:22.018679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.018655    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aadc4c8b-b824-4433-bec7-ab18bc8c2ab2" (UID: "aadc4c8b-b824-4433-bec7-ab18bc8c2ab2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:22.020469 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.020434    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aadc4c8b-b824-4433-bec7-ab18bc8c2ab2" (UID: "aadc4c8b-b824-4433-bec7-ab18bc8c2ab2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:22.020578 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.020551    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-kube-api-access-5dhxg" (OuterVolumeSpecName: "kube-api-access-5dhxg") pod "aadc4c8b-b824-4433-bec7-ab18bc8c2ab2" (UID: "aadc4c8b-b824-4433-bec7-ab18bc8c2ab2"). InnerVolumeSpecName "kube-api-access-5dhxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:22.020713 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.020687    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aadc4c8b-b824-4433-bec7-ab18bc8c2ab2" (UID: "aadc4c8b-b824-4433-bec7-ab18bc8c2ab2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:22.083452 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.083417    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:22.119230 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.119198    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:20:22.119230 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.119225    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:20:22.119230 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.119235    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dhxg\" (UniqueName: \"kubernetes.io/projected/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-kube-api-access-5dhxg\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:20:22.119476 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.119247    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:20:22.445427 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.445374    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:22.542039 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.542008    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:22.572406 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.572340    2554 generic.go:356] "Generic (PLEG): container finished" podID="aadc4c8b-b824-4433-bec7-ab18bc8c2ab2" containerID="a9add5140314a792419a9bc21f05ca944abd0e7d2b535ae90154f714514354e0" exitCode=0
Sep 11 13:20:22.572599 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.572421    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd" event={"ID":"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2","Type":"ContainerDied","Data":"a9add5140314a792419a9bc21f05ca944abd0e7d2b535ae90154f714514354e0"}
Sep 11 13:20:22.572599 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.572436    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"
Sep 11 13:20:22.572599 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.572468    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd" event={"ID":"aadc4c8b-b824-4433-bec7-ab18bc8c2ab2","Type":"ContainerDied","Data":"c8e29b17162edca213d32645b84e81aaf72fc4bd6e4bec790262f67a49229272"}
Sep 11 13:20:22.572599 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.572489    2554 scope.go:117] "RemoveContainer" containerID="a9add5140314a792419a9bc21f05ca944abd0e7d2b535ae90154f714514354e0"
Sep 11 13:20:22.581654 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.581636    2554 scope.go:117] "RemoveContainer" containerID="a9add5140314a792419a9bc21f05ca944abd0e7d2b535ae90154f714514354e0"
Sep 11 13:20:22.581995 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:20:22.581975    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9add5140314a792419a9bc21f05ca944abd0e7d2b535ae90154f714514354e0\": container with ID starting with a9add5140314a792419a9bc21f05ca944abd0e7d2b535ae90154f714514354e0 not found: ID does not exist" containerID="a9add5140314a792419a9bc21f05ca944abd0e7d2b535ae90154f714514354e0"
Sep 11 13:20:22.582042 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.582005    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9add5140314a792419a9bc21f05ca944abd0e7d2b535ae90154f714514354e0"} err="failed to get container status \"a9add5140314a792419a9bc21f05ca944abd0e7d2b535ae90154f714514354e0\": rpc error: code = NotFound desc = could not find container \"a9add5140314a792419a9bc21f05ca944abd0e7d2b535ae90154f714514354e0\": container with ID starting with a9add5140314a792419a9bc21f05ca944abd0e7d2b535ae90154f714514354e0 not found: ID does not exist"
Sep 11 13:20:22.596049 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.595988    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"]
Sep 11 13:20:22.598159 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.598124    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-687dcb39-c8668bf75-s5kqd"]
Sep 11 13:20:22.725553 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:22.725471    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:23.155247 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:23.155179    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aadc4c8b-b824-4433-bec7-ab18bc8c2ab2" path="/var/lib/kubelet/pods/aadc4c8b-b824-4433-bec7-ab18bc8c2ab2/volumes"
Sep 11 13:20:23.188851 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:23.188814    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:23.565723 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:23.565688    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:23.666678 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:23.666646    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:24.096674 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.096631    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"]
Sep 11 13:20:24.097157 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.097133    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aadc4c8b-b824-4433-bec7-ab18bc8c2ab2" containerName="maas-api"
Sep 11 13:20:24.097157 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.097153    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="aadc4c8b-b824-4433-bec7-ab18bc8c2ab2" containerName="maas-api"
Sep 11 13:20:24.097273 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.097244    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="aadc4c8b-b824-4433-bec7-ab18bc8c2ab2" containerName="maas-api"
Sep 11 13:20:24.100042 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.100018    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.102784 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.102761    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 13:20:24.103378 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.103348    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 13:20:24.103495 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.103477    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-783a5c5b-kserve-self-signed-certs\""
Sep 11 13:20:24.103552 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.103538    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2mlv2\""
Sep 11 13:20:24.111160 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.111135    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"]
Sep 11 13:20:24.238644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.238608    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-model-cache\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.238644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.238646    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-home\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.238881 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.238685    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-dshm\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.238881 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.238719    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmvqp\" (UniqueName: \"kubernetes.io/projected/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-kube-api-access-rmvqp\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.238881 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.238844    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-tls-certs\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.239041 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.238979    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-tmp-dir\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.340404 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.340346    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-dshm\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.340404 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.340410    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmvqp\" (UniqueName: \"kubernetes.io/projected/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-kube-api-access-rmvqp\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.340644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.340445    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-tls-certs\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.340644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.340488    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-tmp-dir\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.340644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.340556    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-model-cache\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.340644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.340581    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-home\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.340994 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.340969    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-tmp-dir\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.340994 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.340987    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-model-cache\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.341131 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.341047    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-home\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.342652 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.342634    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-dshm\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.342801 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.342785    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-tls-certs\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.350762 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.350673    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmvqp\" (UniqueName: \"kubernetes.io/projected/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-kube-api-access-rmvqp\") pod \"e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") " pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.391143 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.391112    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:24.410222 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.410187    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:24.666457 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.666356    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:24.683880 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:24.683855    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"]
Sep 11 13:20:24.685707 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:20:24.685679    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod009d045f_1fb1_4291_a7e6_8839ab8fe4b8.slice/crio-4b4cc9748b07792852de4da3310ae4e1b96ef54a73fc7a819bfedd36170a8d0e WatchSource:0}: Error finding container 4b4cc9748b07792852de4da3310ae4e1b96ef54a73fc7a819bfedd36170a8d0e: Status 404 returned error can't find the container with id 4b4cc9748b07792852de4da3310ae4e1b96ef54a73fc7a819bfedd36170a8d0e
Sep 11 13:20:25.590316 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:25.590280    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54" event={"ID":"009d045f-1fb1-4291-a7e6-8839ab8fe4b8","Type":"ContainerStarted","Data":"4b4cc9748b07792852de4da3310ae4e1b96ef54a73fc7a819bfedd36170a8d0e"}
Sep 11 13:20:25.660596 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:25.660553    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:25.887090 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:25.887058    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:25.985649 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:25.985615    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:26.092899 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:26.092734    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:26.196527 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:26.196492    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:26.415704 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:26.415577    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:26.598032 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:26.597998    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54" event={"ID":"009d045f-1fb1-4291-a7e6-8839ab8fe4b8","Type":"ContainerStarted","Data":"2a3a2277a7adc751adfd7195fedb58005ee3380a185e141abe60c1f46df7ebcc"}
Sep 11 13:20:26.638527 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:26.638471    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54" podStartSLOduration=1.540402919 podStartE2EDuration="2.638455509s" totalImagesPullingTime="1.09805259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:20:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:20:26.635866785 +0000 UTC m=+949.953323825" watchObservedRunningTime="2026-09-11 13:20:26.638455509 +0000 UTC m=+949.955912548"
Sep 11 13:20:26.857512 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:26.857466    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:27.539463 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.539410    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"]
Sep 11 13:20:27.542448 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.542425    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:27.544908 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.544878    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-d0b8556b-serving-cert\""
Sep 11 13:20:27.545043 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.544907    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zxpw2\""
Sep 11 13:20:27.545430 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.545408    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 13:20:27.545554 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.545436    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:20:27.545554 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.545451    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 13:20:27.556573 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.556547    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"]
Sep 11 13:20:27.587469 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.587436    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:27.675143 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.675104    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-metrics-certs\") pod \"maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:27.675588 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.675154    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx7h2\" (UniqueName: \"kubernetes.io/projected/e2cc508e-ed59-4167-99cb-77cf951beb4b-kube-api-access-lx7h2\") pod \"maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:27.675588 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.675245    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-maas-api-tls\") pod \"maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:27.675588 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.675298    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2cc508e-ed59-4167-99cb-77cf951beb4b-openshift-service-ca\") pod \"maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:27.688489 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.688461    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:27.776447 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.776411    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-metrics-certs\") pod \"maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:27.776643 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.776470    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx7h2\" (UniqueName: \"kubernetes.io/projected/e2cc508e-ed59-4167-99cb-77cf951beb4b-kube-api-access-lx7h2\") pod \"maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:27.776643 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.776559    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-maas-api-tls\") pod \"maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:27.776643 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.776589    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2cc508e-ed59-4167-99cb-77cf951beb4b-openshift-service-ca\") pod \"maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:27.776808 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:20:27.776667    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-serving-cert: secret "maas-api-e2e-migrate-d0b8556b-serving-cert" not found
Sep 11 13:20:27.776808 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:20:27.776739    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-maas-api-tls podName:e2cc508e-ed59-4167-99cb-77cf951beb4b nodeName:}" failed. No retries permitted until 2026-09-11 13:20:28.276716821 +0000 UTC m=+951.594173871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-maas-api-tls") pod "maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv" (UID: "e2cc508e-ed59-4167-99cb-77cf951beb4b") : secret "maas-api-e2e-migrate-d0b8556b-serving-cert" not found
Sep 11 13:20:27.777282 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.777261    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2cc508e-ed59-4167-99cb-77cf951beb4b-openshift-service-ca\") pod \"maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:27.778904 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.778886    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-metrics-certs\") pod \"maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:27.786534 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.786513    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx7h2\" (UniqueName: \"kubernetes.io/projected/e2cc508e-ed59-4167-99cb-77cf951beb4b-kube-api-access-lx7h2\") pod \"maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:27.833811 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:27.833783    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:28.082330 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:28.082293    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:28.182602 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:28.182520    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:28.281546 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:28.281510    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-maas-api-tls\") pod \"maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:28.283828 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:28.283801    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-maas-api-tls\") pod \"maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:28.456123 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:28.456046    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:28.740517 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:28.740490    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:28.741386 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:28.741349    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"]
Sep 11 13:20:28.743033 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:20:28.743000    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2cc508e_ed59_4167_99cb_77cf951beb4b.slice/crio-189158a568bea257a15c323e9781e0e1852fff15aac38cd60cca08beb8cd39b0 WatchSource:0}: Error finding container 189158a568bea257a15c323e9781e0e1852fff15aac38cd60cca08beb8cd39b0: Status 404 returned error can't find the container with id 189158a568bea257a15c323e9781e0e1852fff15aac38cd60cca08beb8cd39b0
Sep 11 13:20:29.616753 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:29.616719    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv" event={"ID":"e2cc508e-ed59-4167-99cb-77cf951beb4b","Type":"ContainerStarted","Data":"87ce2c4e12804b4822cf3605c148b77807c19e5804ad5f3bcb517b31da8cc491"}
Sep 11 13:20:29.616916 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:29.616779    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv" event={"ID":"e2cc508e-ed59-4167-99cb-77cf951beb4b","Type":"ContainerStarted","Data":"189158a568bea257a15c323e9781e0e1852fff15aac38cd60cca08beb8cd39b0"}
Sep 11 13:20:29.616916 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:29.616811    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:29.617599 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:29.617578    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:29.646954 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:29.646899    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv" podStartSLOduration=2.332533715 podStartE2EDuration="2.646879591s" totalImagesPullingTime="314.345876ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:20:29.646243944 +0000 UTC m=+952.963700985" watchObservedRunningTime="2026-09-11 13:20:29.646879591 +0000 UTC m=+952.964336643"
Sep 11 13:20:30.461429 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:30.461395    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:32.943781 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:32.943746    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:33.110216 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:33.110179    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:34.410828 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:34.410794    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:34.411225 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:34.411031    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:34.423249 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:34.423226    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:34.651903 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:34.651872    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:20:35.626986 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:35.626958    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:50.049919 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:50.049881    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:50.861384 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:50.861337    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:51.903539 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:51.903504    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:52.953211 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:52.953177    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:53.832661 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:53.832632    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:54.575073 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:54.575039    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:55.544129 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:55.544098    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:56.081853 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.081817    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"]
Sep 11 13:20:56.087548 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.087529    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.090767 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.090738    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-345d76\"/\"openshift-service-ca.crt\""
Sep 11 13:20:56.090767 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.090756    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-345d76\"/\"autores-model-345d76-kserve-self-signed-certs\""
Sep 11 13:20:56.090961 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.090772    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-345d76\"/\"kube-root-ca.crt\""
Sep 11 13:20:56.090961 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.090794    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-345d76\"/\"default-dockercfg-hsq6t\""
Sep 11 13:20:56.095793 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.095770    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"]
Sep 11 13:20:56.113704 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.113675    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-tmp-dir\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.113704 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.113708    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-tls-certs\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.113925 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.113734    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-model-cache\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.113925 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.113804    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sps86\" (UniqueName: \"kubernetes.io/projected/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-kube-api-access-sps86\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.113925 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.113844    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-dshm\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.113925 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.113861    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-home\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.200508 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.200472    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:56.214804 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.214764    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-model-cache\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.214971 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.214833    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sps86\" (UniqueName: \"kubernetes.io/projected/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-kube-api-access-sps86\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.214971 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.214868    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-dshm\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.214971 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.214896    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-home\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.215148 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.214992    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-tmp-dir\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.215148 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.215025    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-tls-certs\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.215287 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.215249    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-model-cache\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.215408 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.215385    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-home\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.215502 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.215447    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-tmp-dir\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.217218 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.217197    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-dshm\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.217519 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.217502    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-tls-certs\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.223417 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.223392    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sps86\" (UniqueName: \"kubernetes.io/projected/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-kube-api-access-sps86\") pod \"autores-model-345d76-kserve-7849f5699f-f4w4p\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") " pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.398121 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.398029    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:20:56.434966 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.434931    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:56.676543 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.676515    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"]
Sep 11 13:20:56.678468 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:20:56.678438    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef8cd917_3e7c_4111_b9d7_5cffe9ff47e3.slice/crio-1cf54884829ea4a5f33a6acd2025cd9a144e1bfec7c5dae5bb69b0524bc5d26a WatchSource:0}: Error finding container 1cf54884829ea4a5f33a6acd2025cd9a144e1bfec7c5dae5bb69b0524bc5d26a: Status 404 returned error can't find the container with id 1cf54884829ea4a5f33a6acd2025cd9a144e1bfec7c5dae5bb69b0524bc5d26a
Sep 11 13:20:56.753387 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.753331    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p" event={"ID":"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3","Type":"ContainerStarted","Data":"dbf5c6718c24b74b68f9f67dc9f3c1486225435a09c6adddd859d46437019997"}
Sep 11 13:20:56.753582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.753398    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p" event={"ID":"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3","Type":"ContainerStarted","Data":"1cf54884829ea4a5f33a6acd2025cd9a144e1bfec7c5dae5bb69b0524bc5d26a"}
Sep 11 13:20:56.775481 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:56.775430    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p" podStartSLOduration=0.775413889 podStartE2EDuration="775.413889ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:20:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:20:56.77359804 +0000 UTC m=+980.091055081" watchObservedRunningTime="2026-09-11 13:20:56.775413889 +0000 UTC m=+980.092870928"
Sep 11 13:20:57.012354 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.012265    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"]
Sep 11 13:20:57.012587 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.012561    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv" podUID="e2cc508e-ed59-4167-99cb-77cf951beb4b" containerName="maas-api" containerID="cri-o://87ce2c4e12804b4822cf3605c148b77807c19e5804ad5f3bcb517b31da8cc491" gracePeriod=30
Sep 11 13:20:57.036272 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.036234    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:57.137574 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.137536    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:57.237461 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.237419    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:57.266244 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.266176    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:57.324612 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.324579    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx7h2\" (UniqueName: \"kubernetes.io/projected/e2cc508e-ed59-4167-99cb-77cf951beb4b-kube-api-access-lx7h2\") pod \"e2cc508e-ed59-4167-99cb-77cf951beb4b\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") "
Sep 11 13:20:57.324823 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.324629    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-metrics-certs\") pod \"e2cc508e-ed59-4167-99cb-77cf951beb4b\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") "
Sep 11 13:20:57.324823 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.324701    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-maas-api-tls\") pod \"e2cc508e-ed59-4167-99cb-77cf951beb4b\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") "
Sep 11 13:20:57.324823 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.324807    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2cc508e-ed59-4167-99cb-77cf951beb4b-openshift-service-ca\") pod \"e2cc508e-ed59-4167-99cb-77cf951beb4b\" (UID: \"e2cc508e-ed59-4167-99cb-77cf951beb4b\") "
Sep 11 13:20:57.325251 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.325178    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2cc508e-ed59-4167-99cb-77cf951beb4b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e2cc508e-ed59-4167-99cb-77cf951beb4b" (UID: "e2cc508e-ed59-4167-99cb-77cf951beb4b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:57.326859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.326827    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e2cc508e-ed59-4167-99cb-77cf951beb4b" (UID: "e2cc508e-ed59-4167-99cb-77cf951beb4b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:57.326966 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.326877    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e2cc508e-ed59-4167-99cb-77cf951beb4b" (UID: "e2cc508e-ed59-4167-99cb-77cf951beb4b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:57.326966 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.326907    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2cc508e-ed59-4167-99cb-77cf951beb4b-kube-api-access-lx7h2" (OuterVolumeSpecName: "kube-api-access-lx7h2") pod "e2cc508e-ed59-4167-99cb-77cf951beb4b" (UID: "e2cc508e-ed59-4167-99cb-77cf951beb4b"). InnerVolumeSpecName "kube-api-access-lx7h2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:57.341047 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.340942    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:57.415188 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.415155    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:57.426437 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.426407    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2cc508e-ed59-4167-99cb-77cf951beb4b-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:20:57.426437 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.426433    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx7h2\" (UniqueName: \"kubernetes.io/projected/e2cc508e-ed59-4167-99cb-77cf951beb4b-kube-api-access-lx7h2\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:20:57.426437 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.426444    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:20:57.426649 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.426453    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2cc508e-ed59-4167-99cb-77cf951beb4b-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:20:57.653411 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.653349    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:57.759359 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.759326    2554 generic.go:356] "Generic (PLEG): container finished" podID="e2cc508e-ed59-4167-99cb-77cf951beb4b" containerID="87ce2c4e12804b4822cf3605c148b77807c19e5804ad5f3bcb517b31da8cc491" exitCode=0
Sep 11 13:20:57.759542 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.759383    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"
Sep 11 13:20:57.759542 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.759417    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv" event={"ID":"e2cc508e-ed59-4167-99cb-77cf951beb4b","Type":"ContainerDied","Data":"87ce2c4e12804b4822cf3605c148b77807c19e5804ad5f3bcb517b31da8cc491"}
Sep 11 13:20:57.759542 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.759481    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv" event={"ID":"e2cc508e-ed59-4167-99cb-77cf951beb4b","Type":"ContainerDied","Data":"189158a568bea257a15c323e9781e0e1852fff15aac38cd60cca08beb8cd39b0"}
Sep 11 13:20:57.759542 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.759511    2554 scope.go:117] "RemoveContainer" containerID="87ce2c4e12804b4822cf3605c148b77807c19e5804ad5f3bcb517b31da8cc491"
Sep 11 13:20:57.769416 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.769395    2554 scope.go:117] "RemoveContainer" containerID="87ce2c4e12804b4822cf3605c148b77807c19e5804ad5f3bcb517b31da8cc491"
Sep 11 13:20:57.769697 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:20:57.769680    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87ce2c4e12804b4822cf3605c148b77807c19e5804ad5f3bcb517b31da8cc491\": container with ID starting with 87ce2c4e12804b4822cf3605c148b77807c19e5804ad5f3bcb517b31da8cc491 not found: ID does not exist" containerID="87ce2c4e12804b4822cf3605c148b77807c19e5804ad5f3bcb517b31da8cc491"
Sep 11 13:20:57.769745 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.769707    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87ce2c4e12804b4822cf3605c148b77807c19e5804ad5f3bcb517b31da8cc491"} err="failed to get container status \"87ce2c4e12804b4822cf3605c148b77807c19e5804ad5f3bcb517b31da8cc491\": rpc error: code = NotFound desc = could not find container \"87ce2c4e12804b4822cf3605c148b77807c19e5804ad5f3bcb517b31da8cc491\": container with ID starting with 87ce2c4e12804b4822cf3605c148b77807c19e5804ad5f3bcb517b31da8cc491 not found: ID does not exist"
Sep 11 13:20:57.783852 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.783822    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"]
Sep 11 13:20:57.789013 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.788981    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0b8556b-56c67847bd-bc8tv"]
Sep 11 13:20:57.815818 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:57.815790    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:58.144277 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:58.144244    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:58.706169 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:58.706135    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:58.853431 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:58.853393    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:20:59.153865 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:59.153832    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2cc508e-ed59-4167-99cb-77cf951beb4b" path="/var/lib/kubelet/pods/e2cc508e-ed59-4167-99cb-77cf951beb4b/volumes"
Sep 11 13:20:59.669574 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:20:59.669537    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:00.608218 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:00.608190    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:01.060174 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:01.060071    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:01.169938 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:01.169904    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:01.782532 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:01.782499    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:02.544310 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:02.544261    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:02.662577 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:02.662540    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:03.456986 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:03.456954    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:04.295430 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.295398    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"]
Sep 11 13:21:04.295876 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.295853    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2cc508e-ed59-4167-99cb-77cf951beb4b" containerName="maas-api"
Sep 11 13:21:04.295876 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.295875    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2cc508e-ed59-4167-99cb-77cf951beb4b" containerName="maas-api"
Sep 11 13:21:04.296019 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.295955    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2cc508e-ed59-4167-99cb-77cf951beb4b" containerName="maas-api"
Sep 11 13:21:04.298730 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.298708    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:04.301683 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.301661    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:21:04.302244 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.302195    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zxpw2\""
Sep 11 13:21:04.302244 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.302202    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-cd28402c-serving-cert\""
Sep 11 13:21:04.302393 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.302249    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 13:21:04.302459 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.302389    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 13:21:04.311163 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.311129    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"]
Sep 11 13:21:04.329716 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.329676    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:04.388504 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.388469    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-maas-api-tls\") pod \"maas-api-e2e-ait-cd28402c-754996b87c-78s9g\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:04.388661 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.388537    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80433f98-d7b6-4437-966d-b476e64d9793-openshift-service-ca\") pod \"maas-api-e2e-ait-cd28402c-754996b87c-78s9g\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:04.388661 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.388578    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5dp9\" (UniqueName: \"kubernetes.io/projected/80433f98-d7b6-4437-966d-b476e64d9793-kube-api-access-q5dp9\") pod \"maas-api-e2e-ait-cd28402c-754996b87c-78s9g\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:04.388661 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.388595    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-metrics-certs\") pod \"maas-api-e2e-ait-cd28402c-754996b87c-78s9g\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:04.428774 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.428740    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:04.489611 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.489578    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80433f98-d7b6-4437-966d-b476e64d9793-openshift-service-ca\") pod \"maas-api-e2e-ait-cd28402c-754996b87c-78s9g\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:04.490087 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.489629    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5dp9\" (UniqueName: \"kubernetes.io/projected/80433f98-d7b6-4437-966d-b476e64d9793-kube-api-access-q5dp9\") pod \"maas-api-e2e-ait-cd28402c-754996b87c-78s9g\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:04.490087 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.489647    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-metrics-certs\") pod \"maas-api-e2e-ait-cd28402c-754996b87c-78s9g\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:04.490087 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.489691    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-maas-api-tls\") pod \"maas-api-e2e-ait-cd28402c-754996b87c-78s9g\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:04.490087 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:21:04.489800    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-serving-cert: secret "maas-api-e2e-ait-cd28402c-serving-cert" not found
Sep 11 13:21:04.490087 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:21:04.489874    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-maas-api-tls podName:80433f98-d7b6-4437-966d-b476e64d9793 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:04.989852953 +0000 UTC m=+988.307309970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-maas-api-tls") pod "maas-api-e2e-ait-cd28402c-754996b87c-78s9g" (UID: "80433f98-d7b6-4437-966d-b476e64d9793") : secret "maas-api-e2e-ait-cd28402c-serving-cert" not found
Sep 11 13:21:04.490490 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.490468    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80433f98-d7b6-4437-966d-b476e64d9793-openshift-service-ca\") pod \"maas-api-e2e-ait-cd28402c-754996b87c-78s9g\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:04.492236 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.492214    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-metrics-certs\") pod \"maas-api-e2e-ait-cd28402c-754996b87c-78s9g\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:04.498672 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.498637    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5dp9\" (UniqueName: \"kubernetes.io/projected/80433f98-d7b6-4437-966d-b476e64d9793-kube-api-access-q5dp9\") pod \"maas-api-e2e-ait-cd28402c-754996b87c-78s9g\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:04.674517 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.674482    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:04.864347 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.864311    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:04.976762 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.976680    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:04.994964 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.994928    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-maas-api-tls\") pod \"maas-api-e2e-ait-cd28402c-754996b87c-78s9g\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:05.000338 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:04.997535    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-maas-api-tls\") pod \"maas-api-e2e-ait-cd28402c-754996b87c-78s9g\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:05.210813 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:05.210779    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:05.483315 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:05.483287    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"]
Sep 11 13:21:05.485219 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:21:05.485193    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80433f98_d7b6_4437_966d_b476e64d9793.slice/crio-6990a9faaaad849d93a29eccd1f435bca7e325d6bfd6937fc61a1bc8d20f5595 WatchSource:0}: Error finding container 6990a9faaaad849d93a29eccd1f435bca7e325d6bfd6937fc61a1bc8d20f5595: Status 404 returned error can't find the container with id 6990a9faaaad849d93a29eccd1f435bca7e325d6bfd6937fc61a1bc8d20f5595
Sep 11 13:21:05.805530 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:05.805490    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g" event={"ID":"80433f98-d7b6-4437-966d-b476e64d9793","Type":"ContainerStarted","Data":"6990a9faaaad849d93a29eccd1f435bca7e325d6bfd6937fc61a1bc8d20f5595"}
Sep 11 13:21:06.398505 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:06.398460    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:21:06.398764 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:06.398517    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:21:06.412753 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:06.412722    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:21:06.812584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:06.812485    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g" event={"ID":"80433f98-d7b6-4437-966d-b476e64d9793","Type":"ContainerStarted","Data":"8b6eb059ba4e5fa00e76a14e93aa5a5469ffa298f006e8c6f9237a381fb4a950"}
Sep 11 13:21:06.823971 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:06.823944    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:21:06.832388 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:06.832252    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g" podStartSLOduration=2.253856984 podStartE2EDuration="2.832233062s" totalImagesPullingTime="578.376078ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:06.831415905 +0000 UTC m=+990.148872947" watchObservedRunningTime="2026-09-11 13:21:06.832233062 +0000 UTC m=+990.149690103"
Sep 11 13:21:07.817452 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:07.817422    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:10.182062 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.182027    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"]
Sep 11 13:21:10.185437 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.185420    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.188034 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.188013    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a649dc\"/\"openshift-service-ca.crt\""
Sep 11 13:21:10.188570 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.188553    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a649dc\"/\"kube-root-ca.crt\""
Sep 11 13:21:10.188651 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.188632    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a649dc\"/\"default-dockercfg-ndpnf\""
Sep 11 13:21:10.188718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.188695    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a649dc\"/\"test-model-a649dc-kserve-self-signed-certs\""
Sep 11 13:21:10.199890 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.199867    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"]
Sep 11 13:21:10.244572 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.244538    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ms2c\" (UniqueName: \"kubernetes.io/projected/65646e1b-5b96-43bd-9f67-53b41bc86a95-kube-api-access-7ms2c\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.244739 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.244579    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-dshm\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.244739 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.244598    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65646e1b-5b96-43bd-9f67-53b41bc86a95-tls-certs\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.244739 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.244658    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-home\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.244739 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.244695    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-tmp-dir\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.244739 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.244715    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-model-cache\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.345630 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.345590    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ms2c\" (UniqueName: \"kubernetes.io/projected/65646e1b-5b96-43bd-9f67-53b41bc86a95-kube-api-access-7ms2c\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.345630 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.345633    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-dshm\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.345877 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.345651    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65646e1b-5b96-43bd-9f67-53b41bc86a95-tls-certs\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.345877 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.345686    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-home\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.345877 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.345726    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-tmp-dir\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.345877 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.345750    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-model-cache\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.346147 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.346124    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-tmp-dir\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.346206 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.346163    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-home\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.346206 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.346183    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-model-cache\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.347845 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.347826    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-dshm\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.348094 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.348078    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65646e1b-5b96-43bd-9f67-53b41bc86a95-tls-certs\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.354027 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.354006    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ms2c\" (UniqueName: \"kubernetes.io/projected/65646e1b-5b96-43bd-9f67-53b41bc86a95-kube-api-access-7ms2c\") pod \"test-model-a649dc-kserve-768bc778c5-4fqhz\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") " pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.494739 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.494639    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:10.583802 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.583760    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:10.768067 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.767989    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"]
Sep 11 13:21:10.770533 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:21:10.770496    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65646e1b_5b96_43bd_9f67_53b41bc86a95.slice/crio-6719d324d53baa6dd9dde6fc286e458513709508a5245b3f11adb54bdbadaa27 WatchSource:0}: Error finding container 6719d324d53baa6dd9dde6fc286e458513709508a5245b3f11adb54bdbadaa27: Status 404 returned error can't find the container with id 6719d324d53baa6dd9dde6fc286e458513709508a5245b3f11adb54bdbadaa27
Sep 11 13:21:10.833747 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:10.833711    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz" event={"ID":"65646e1b-5b96-43bd-9f67-53b41bc86a95","Type":"ContainerStarted","Data":"6719d324d53baa6dd9dde6fc286e458513709508a5245b3f11adb54bdbadaa27"}
Sep 11 13:21:11.840195 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:11.840156    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz" event={"ID":"65646e1b-5b96-43bd-9f67-53b41bc86a95","Type":"ContainerStarted","Data":"9e6c98f122924e3f6d3641019bfd28cb653db67bb5d68c9de2e9294681b18cd2"}
Sep 11 13:21:11.862321 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:11.862263    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz" podStartSLOduration=1.862247484 podStartE2EDuration="1.862247484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:11.860342032 +0000 UTC m=+995.177799073" watchObservedRunningTime="2026-09-11 13:21:11.862247484 +0000 UTC m=+995.179704523"
Sep 11 13:21:13.828130 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:13.828096    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:19.052745 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:19.052708    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:20.494783 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:20.494730    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:20.494783 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:20.494789    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:20.507493 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:20.507460    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:20.899200 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:20.899162    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:21:25.276693 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:25.276660    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:27.157941 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:27.157911    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:27.904329 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:27.904295    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:28.604840 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:28.604806    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:29.362803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:29.362767    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:30.098701 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:30.098667    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:31.714041 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.714010    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"]
Sep 11 13:21:31.717779 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.717757    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.719846 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.719827    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-783a5c5b-kserve-self-signed-certs\""
Sep 11 13:21:31.732878 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.732853    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"]
Sep 11 13:21:31.737922 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.737902    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:31.748217 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.748182    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-model-cache\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.748318 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.748251    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/510ce1f2-803a-433e-930d-b6af6e59ff22-tls-certs\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.748318 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.748312    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-tmp-dir\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.748444 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.748343    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-dshm\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.748444 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.748401    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v59r9\" (UniqueName: \"kubernetes.io/projected/510ce1f2-803a-433e-930d-b6af6e59ff22-kube-api-access-v59r9\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.748521 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.748481    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-home\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.849489 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.849456    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-model-cache\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.849644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.849507    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/510ce1f2-803a-433e-930d-b6af6e59ff22-tls-certs\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.849644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.849539    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-tmp-dir\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.849644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.849557    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-dshm\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.849644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.849578    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v59r9\" (UniqueName: \"kubernetes.io/projected/510ce1f2-803a-433e-930d-b6af6e59ff22-kube-api-access-v59r9\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.849644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.849629    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-home\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.849947 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.849921    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-model-cache\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.850020 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.849922    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-tmp-dir\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.850020 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.850004    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-home\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.851854 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.851828    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-dshm\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.852073 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.852055    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/510ce1f2-803a-433e-930d-b6af6e59ff22-tls-certs\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:31.857182 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:31.857161    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v59r9\" (UniqueName: \"kubernetes.io/projected/510ce1f2-803a-433e-930d-b6af6e59ff22-kube-api-access-v59r9\") pod \"e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") " pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:32.027225 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:32.027140    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:32.302697 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:32.302672    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"]
Sep 11 13:21:32.304469 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:21:32.304433    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod510ce1f2_803a_433e_930d_b6af6e59ff22.slice/crio-6cf101a962aa5d321a8b4e185b6c3b9ae8fb4b4dff17faa43302ff7427bcf1ad WatchSource:0}: Error finding container 6cf101a962aa5d321a8b4e185b6c3b9ae8fb4b4dff17faa43302ff7427bcf1ad: Status 404 returned error can't find the container with id 6cf101a962aa5d321a8b4e185b6c3b9ae8fb4b4dff17faa43302ff7427bcf1ad
Sep 11 13:21:32.949465 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:32.949413    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp" event={"ID":"510ce1f2-803a-433e-930d-b6af6e59ff22","Type":"ContainerStarted","Data":"600fb5cc24816af6597baa2ec53911ac169d0263cfc63b7e4bfae34dfb76d947"}
Sep 11 13:21:32.949465 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:32.949473    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp" event={"ID":"510ce1f2-803a-433e-930d-b6af6e59ff22","Type":"ContainerStarted","Data":"6cf101a962aa5d321a8b4e185b6c3b9ae8fb4b4dff17faa43302ff7427bcf1ad"}
Sep 11 13:21:32.958445 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:32.958415    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:32.981571 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:32.981509    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp" podStartSLOduration=1.981488829 podStartE2EDuration="1.981488829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:32.978805184 +0000 UTC m=+1016.296262225" watchObservedRunningTime="2026-09-11 13:21:32.981488829 +0000 UTC m=+1016.298945870"
Sep 11 13:21:33.955241 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:33.955204    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:34.720102 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:34.720062    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:35.685436 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:35.685402    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"]
Sep 11 13:21:35.702541 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:35.702507    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:35.801886 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:35.801854    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:36.487311 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:36.487277    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:42.019816 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:42.019779    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:42.027809 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:42.027787    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:42.027948 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:42.027930    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:42.040382 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:42.040351    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:42.680722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:42.680692    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"]
Sep 11 13:21:42.680932 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:42.680904    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g" podUID="80433f98-d7b6-4437-966d-b476e64d9793" containerName="maas-api" containerID="cri-o://8b6eb059ba4e5fa00e76a14e93aa5a5469ffa298f006e8c6f9237a381fb4a950" gracePeriod=30
Sep 11 13:21:42.699279 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:42.699254    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:42.794083 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:42.794047    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:42.880091 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:42.880056    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:42.942257 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:42.942232    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:42.960581 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:42.960549    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:43.004649 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.004620    2554 generic.go:356] "Generic (PLEG): container finished" podID="80433f98-d7b6-4437-966d-b476e64d9793" containerID="8b6eb059ba4e5fa00e76a14e93aa5a5469ffa298f006e8c6f9237a381fb4a950" exitCode=0
Sep 11 13:21:43.004813 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.004664    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"
Sep 11 13:21:43.004813 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.004706    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g" event={"ID":"80433f98-d7b6-4437-966d-b476e64d9793","Type":"ContainerDied","Data":"8b6eb059ba4e5fa00e76a14e93aa5a5469ffa298f006e8c6f9237a381fb4a950"}
Sep 11 13:21:43.004813 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.004769    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g" event={"ID":"80433f98-d7b6-4437-966d-b476e64d9793","Type":"ContainerDied","Data":"6990a9faaaad849d93a29eccd1f435bca7e325d6bfd6937fc61a1bc8d20f5595"}
Sep 11 13:21:43.004813 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.004798    2554 scope.go:117] "RemoveContainer" containerID="8b6eb059ba4e5fa00e76a14e93aa5a5469ffa298f006e8c6f9237a381fb4a950"
Sep 11 13:21:43.014071 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.014054    2554 scope.go:117] "RemoveContainer" containerID="8b6eb059ba4e5fa00e76a14e93aa5a5469ffa298f006e8c6f9237a381fb4a950"
Sep 11 13:21:43.014311 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:21:43.014294    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b6eb059ba4e5fa00e76a14e93aa5a5469ffa298f006e8c6f9237a381fb4a950\": container with ID starting with 8b6eb059ba4e5fa00e76a14e93aa5a5469ffa298f006e8c6f9237a381fb4a950 not found: ID does not exist" containerID="8b6eb059ba4e5fa00e76a14e93aa5a5469ffa298f006e8c6f9237a381fb4a950"
Sep 11 13:21:43.014388 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.014321    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b6eb059ba4e5fa00e76a14e93aa5a5469ffa298f006e8c6f9237a381fb4a950"} err="failed to get container status \"8b6eb059ba4e5fa00e76a14e93aa5a5469ffa298f006e8c6f9237a381fb4a950\": rpc error: code = NotFound desc = could not find container \"8b6eb059ba4e5fa00e76a14e93aa5a5469ffa298f006e8c6f9237a381fb4a950\": container with ID starting with 8b6eb059ba4e5fa00e76a14e93aa5a5469ffa298f006e8c6f9237a381fb4a950 not found: ID does not exist"
Sep 11 13:21:43.016860 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.016839    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:21:43.051298 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.051266    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-metrics-certs\") pod \"80433f98-d7b6-4437-966d-b476e64d9793\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") "
Sep 11 13:21:43.051720 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.051321    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80433f98-d7b6-4437-966d-b476e64d9793-openshift-service-ca\") pod \"80433f98-d7b6-4437-966d-b476e64d9793\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") "
Sep 11 13:21:43.051720 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.051348    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-maas-api-tls\") pod \"80433f98-d7b6-4437-966d-b476e64d9793\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") "
Sep 11 13:21:43.051720 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.051396    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5dp9\" (UniqueName: \"kubernetes.io/projected/80433f98-d7b6-4437-966d-b476e64d9793-kube-api-access-q5dp9\") pod \"80433f98-d7b6-4437-966d-b476e64d9793\" (UID: \"80433f98-d7b6-4437-966d-b476e64d9793\") "
Sep 11 13:21:43.051856 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.051732    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80433f98-d7b6-4437-966d-b476e64d9793-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "80433f98-d7b6-4437-966d-b476e64d9793" (UID: "80433f98-d7b6-4437-966d-b476e64d9793"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:21:43.053660 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.053635    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "80433f98-d7b6-4437-966d-b476e64d9793" (UID: "80433f98-d7b6-4437-966d-b476e64d9793"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:21:43.053754 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.053660    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "80433f98-d7b6-4437-966d-b476e64d9793" (UID: "80433f98-d7b6-4437-966d-b476e64d9793"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:21:43.053754 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.053686    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80433f98-d7b6-4437-966d-b476e64d9793-kube-api-access-q5dp9" (OuterVolumeSpecName: "kube-api-access-q5dp9") pod "80433f98-d7b6-4437-966d-b476e64d9793" (UID: "80433f98-d7b6-4437-966d-b476e64d9793"). InnerVolumeSpecName "kube-api-access-q5dp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:21:43.089621 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.089592    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:43.152793 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.152765    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.152793 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.152790    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80433f98-d7b6-4437-966d-b476e64d9793-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.152793 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.152800    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80433f98-d7b6-4437-966d-b476e64d9793-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.153082 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.152809    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5dp9\" (UniqueName: \"kubernetes.io/projected/80433f98-d7b6-4437-966d-b476e64d9793-kube-api-access-q5dp9\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.184118 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.184046    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:43.307656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.307610    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:43.322834 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.322802    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"]
Sep 11 13:21:43.327253 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.327229    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cd28402c-754996b87c-78s9g"]
Sep 11 13:21:43.569189 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.569155    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:43.664271 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.664242    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:43.738991 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:43.738958    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:44.158266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:44.158240    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:44.459215 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:44.459131    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:44.632337 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:44.632306    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:44.985021 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:44.984988    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:45.082118 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:45.082085    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:45.153163 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:45.153128    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80433f98-d7b6-4437-966d-b476e64d9793" path="/var/lib/kubelet/pods/80433f98-d7b6-4437-966d-b476e64d9793/volumes"
Sep 11 13:21:45.345007 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:45.344981    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:46.445403 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:46.445345    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:47.075250 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:47.075216    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:47.241469 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:47.241438    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:47.683470 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:47.683436    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:48.733907 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:48.733873    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:48.836512 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:48.836477    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:49.542424 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:49.542392    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:50.264675 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.264635    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"]
Sep 11 13:21:50.265196 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.265179    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80433f98-d7b6-4437-966d-b476e64d9793" containerName="maas-api"
Sep 11 13:21:50.265256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.265199    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="80433f98-d7b6-4437-966d-b476e64d9793" containerName="maas-api"
Sep 11 13:21:50.265293 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.265267    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="80433f98-d7b6-4437-966d-b476e64d9793" containerName="maas-api"
Sep 11 13:21:50.269795 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.269771    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.272144 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.272125    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-adc38cd1-serving-cert\""
Sep 11 13:21:50.272453 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.272420    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 13:21:50.272704 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.272686    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:21:50.272704 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.272695    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 13:21:50.272844 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.272706    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zxpw2\""
Sep 11 13:21:50.280771 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.280736    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"]
Sep 11 13:21:50.292588 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.292551    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:50.393635 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.393605    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:50.416927 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.416890    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6f30b52-e5d5-41f9-abd0-affd163c68f9-openshift-service-ca\") pod \"maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.417105 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.416945    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6f30b52-e5d5-41f9-abd0-affd163c68f9-maas-api-tls\") pod \"maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.417105 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.416975    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59jrk\" (UniqueName: \"kubernetes.io/projected/f6f30b52-e5d5-41f9-abd0-affd163c68f9-kube-api-access-59jrk\") pod \"maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.417105 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.417076    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f30b52-e5d5-41f9-abd0-affd163c68f9-metrics-certs\") pod \"maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.463947 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.463908    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:50.518486 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.518359    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f30b52-e5d5-41f9-abd0-affd163c68f9-metrics-certs\") pod \"maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.518486 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.518460    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6f30b52-e5d5-41f9-abd0-affd163c68f9-openshift-service-ca\") pod \"maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.518709 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.518503    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6f30b52-e5d5-41f9-abd0-affd163c68f9-maas-api-tls\") pod \"maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.518709 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.518541    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59jrk\" (UniqueName: \"kubernetes.io/projected/f6f30b52-e5d5-41f9-abd0-affd163c68f9-kube-api-access-59jrk\") pod \"maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.519185 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.519149    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6f30b52-e5d5-41f9-abd0-affd163c68f9-openshift-service-ca\") pod \"maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.520944 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.520917    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6f30b52-e5d5-41f9-abd0-affd163c68f9-maas-api-tls\") pod \"maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.521058 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.521038    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f30b52-e5d5-41f9-abd0-affd163c68f9-metrics-certs\") pod \"maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.527340 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.527309    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59jrk\" (UniqueName: \"kubernetes.io/projected/f6f30b52-e5d5-41f9-abd0-affd163c68f9-kube-api-access-59jrk\") pod \"maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.581004 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.580970    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:50.710545 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.710496    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p" podUID="ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3" containerName="main" containerID="cri-o://dbf5c6718c24b74b68f9f67dc9f3c1486225435a09c6adddd859d46437019997" gracePeriod=45
Sep 11 13:21:50.808217 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.808086    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:50.862879 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.862852    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"]
Sep 11 13:21:50.866731 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:21:50.866674    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f30b52_e5d5_41f9_abd0_affd163c68f9.slice/crio-f9e1b79330d75c39bf5d6072ffa7a21a63c0c872e18ed5341c6e95a3e835e6cd WatchSource:0}: Error finding container f9e1b79330d75c39bf5d6072ffa7a21a63c0c872e18ed5341c6e95a3e835e6cd: Status 404 returned error can't find the container with id f9e1b79330d75c39bf5d6072ffa7a21a63c0c872e18ed5341c6e95a3e835e6cd
Sep 11 13:21:50.907303 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.907271    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:50.959108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:50.959084    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:21:51.023468 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.023434    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sps86\" (UniqueName: \"kubernetes.io/projected/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-kube-api-access-sps86\") pod \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") "
Sep 11 13:21:51.023656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.023479    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-dshm\") pod \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") "
Sep 11 13:21:51.023656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.023518    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-tmp-dir\") pod \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") "
Sep 11 13:21:51.023656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.023532    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-model-cache\") pod \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") "
Sep 11 13:21:51.023656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.023572    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-home\") pod \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") "
Sep 11 13:21:51.023656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.023618    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-tls-certs\") pod \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\" (UID: \"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3\") "
Sep 11 13:21:51.023920 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.023828    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-model-cache" (OuterVolumeSpecName: "model-cache") pod "ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3" (UID: "ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:51.023920 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.023842    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3" (UID: "ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:51.024016 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.023913    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-home" (OuterVolumeSpecName: "home") pod "ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3" (UID: "ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:51.024072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.024024    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:21:51.024072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.024043    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:21:51.024072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.024057    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:21:51.025857 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.025836    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3" (UID: "ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:21:51.026041 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.026018    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-kube-api-access-sps86" (OuterVolumeSpecName: "kube-api-access-sps86") pod "ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3" (UID: "ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3"). InnerVolumeSpecName "kube-api-access-sps86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:21:51.026041 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.026023    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-dshm" (OuterVolumeSpecName: "dshm") pod "ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3" (UID: "ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:51.052532 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.052501    2554 generic.go:356] "Generic (PLEG): container finished" podID="ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3" containerID="dbf5c6718c24b74b68f9f67dc9f3c1486225435a09c6adddd859d46437019997" exitCode=0
Sep 11 13:21:51.052709 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.052565    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"
Sep 11 13:21:51.052709 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.052588    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p" event={"ID":"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3","Type":"ContainerDied","Data":"dbf5c6718c24b74b68f9f67dc9f3c1486225435a09c6adddd859d46437019997"}
Sep 11 13:21:51.052709 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.052616    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p" event={"ID":"ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3","Type":"ContainerDied","Data":"1cf54884829ea4a5f33a6acd2025cd9a144e1bfec7c5dae5bb69b0524bc5d26a"}
Sep 11 13:21:51.052709 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.052635    2554 scope.go:117] "RemoveContainer" containerID="dbf5c6718c24b74b68f9f67dc9f3c1486225435a09c6adddd859d46437019997"
Sep 11 13:21:51.054634 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.054607    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt" event={"ID":"f6f30b52-e5d5-41f9-abd0-affd163c68f9","Type":"ContainerStarted","Data":"f9e1b79330d75c39bf5d6072ffa7a21a63c0c872e18ed5341c6e95a3e835e6cd"}
Sep 11 13:21:51.063178 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.063159    2554 scope.go:117] "RemoveContainer" containerID="dbf5c6718c24b74b68f9f67dc9f3c1486225435a09c6adddd859d46437019997"
Sep 11 13:21:51.063549 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:21:51.063525    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbf5c6718c24b74b68f9f67dc9f3c1486225435a09c6adddd859d46437019997\": container with ID starting with dbf5c6718c24b74b68f9f67dc9f3c1486225435a09c6adddd859d46437019997 not found: ID does not exist" containerID="dbf5c6718c24b74b68f9f67dc9f3c1486225435a09c6adddd859d46437019997"
Sep 11 13:21:51.063635 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.063556    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbf5c6718c24b74b68f9f67dc9f3c1486225435a09c6adddd859d46437019997"} err="failed to get container status \"dbf5c6718c24b74b68f9f67dc9f3c1486225435a09c6adddd859d46437019997\": rpc error: code = NotFound desc = could not find container \"dbf5c6718c24b74b68f9f67dc9f3c1486225435a09c6adddd859d46437019997\": container with ID starting with dbf5c6718c24b74b68f9f67dc9f3c1486225435a09c6adddd859d46437019997 not found: ID does not exist"
Sep 11 13:21:51.078468 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.078430    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"]
Sep 11 13:21:51.082052 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.082022    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-345d76/autores-model-345d76-kserve-7849f5699f-f4w4p"]
Sep 11 13:21:51.124530 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.124494    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sps86\" (UniqueName: \"kubernetes.io/projected/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-kube-api-access-sps86\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:21:51.124530 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.124527    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:21:51.124724 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.124544    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:21:51.154066 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:51.154035    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3" path="/var/lib/kubelet/pods/ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3/volumes"
Sep 11 13:21:52.062332 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:52.062295    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt" event={"ID":"f6f30b52-e5d5-41f9-abd0-affd163c68f9","Type":"ContainerStarted","Data":"7070231e47f8a1c25c349d979d996b3bfa79c6b02537337c5550462b6fc89643"}
Sep 11 13:21:52.062758 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:52.062356    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:52.085834 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:52.085780    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt" podStartSLOduration=1.6322913909999999 podStartE2EDuration="2.085763992s" totalImagesPullingTime="453.472601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:52.083010549 +0000 UTC m=+1035.400467589" watchObservedRunningTime="2026-09-11 13:21:52.085763992 +0000 UTC m=+1035.403221031"
Sep 11 13:21:53.191378 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:53.191331    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"]
Sep 11 13:21:53.213700 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:53.213670    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:55.931881 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:55.931840    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:56.203092 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:56.203003    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:56.631774 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:56.631737    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:56.992118 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:56.992033    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:57.421074 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:57.421039    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:58.073213 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:58.073183    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:21:59.046814 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:59.046778    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:59.801270 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:21:59.801241    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:00.457210 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:00.457174    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:00.613066 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:00.613031    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:01.214478 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:01.214443    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:02.327765 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:02.327730    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:03.668944 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.668911    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"]
Sep 11 13:22:03.669305 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.669256    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3" containerName="main"
Sep 11 13:22:03.669305 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.669268    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3" containerName="main"
Sep 11 13:22:03.669409 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.669327    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef8cd917-3e7c-4111-b9d7-5cffe9ff47e3" containerName="main"
Sep 11 13:22:03.672813 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.672788    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:03.675187 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.675161    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-1c07a9-serving-cert\""
Sep 11 13:22:03.689941 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.689918    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"]
Sep 11 13:22:03.716480 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.716326    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:03.837718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.837686    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/700c665f-9faf-424c-b571-24b021cf4554-maas-api-tls\") pod \"maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:03.837916 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.837737    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whxqx\" (UniqueName: \"kubernetes.io/projected/700c665f-9faf-424c-b571-24b021cf4554-kube-api-access-whxqx\") pod \"maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:03.837916 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.837811    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/700c665f-9faf-424c-b571-24b021cf4554-openshift-service-ca\") pod \"maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:03.837916 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.837857    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/700c665f-9faf-424c-b571-24b021cf4554-metrics-certs\") pod \"maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:03.840002 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.839978    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:03.938619 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.938543    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/700c665f-9faf-424c-b571-24b021cf4554-maas-api-tls\") pod \"maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:03.938619 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.938598    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whxqx\" (UniqueName: \"kubernetes.io/projected/700c665f-9faf-424c-b571-24b021cf4554-kube-api-access-whxqx\") pod \"maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:03.938799 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.938650    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/700c665f-9faf-424c-b571-24b021cf4554-openshift-service-ca\") pod \"maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:03.938799 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.938682    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/700c665f-9faf-424c-b571-24b021cf4554-metrics-certs\") pod \"maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:03.939302 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.939272    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/700c665f-9faf-424c-b571-24b021cf4554-openshift-service-ca\") pod \"maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:03.941174 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.941145    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/700c665f-9faf-424c-b571-24b021cf4554-maas-api-tls\") pod \"maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:03.941243 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.941189    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/700c665f-9faf-424c-b571-24b021cf4554-metrics-certs\") pod \"maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:03.944022 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.943998    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:03.948256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.948235    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whxqx\" (UniqueName: \"kubernetes.io/projected/700c665f-9faf-424c-b571-24b021cf4554-kube-api-access-whxqx\") pod \"maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:03.983081 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:03.983055    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:04.281872 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:04.281807    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"]
Sep 11 13:22:04.284227 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:22:04.284199    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod700c665f_9faf_424c_b571_24b021cf4554.slice/crio-4ad21517fd1025e02de4ee74cdab9835142f760085b1d7e3f61a957d3f30944d WatchSource:0}: Error finding container 4ad21517fd1025e02de4ee74cdab9835142f760085b1d7e3f61a957d3f30944d: Status 404 returned error can't find the container with id 4ad21517fd1025e02de4ee74cdab9835142f760085b1d7e3f61a957d3f30944d
Sep 11 13:22:04.572062 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:04.572034    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:04.697105 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:04.697074    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:05.129842 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:05.129808    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt" event={"ID":"700c665f-9faf-424c-b571-24b021cf4554","Type":"ContainerStarted","Data":"a0d5069dcdae2cc1e7317084a423e5810dc516fdad11aec68243037d70c8f36d"}
Sep 11 13:22:05.130077 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:05.129866    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt" event={"ID":"700c665f-9faf-424c-b571-24b021cf4554","Type":"ContainerStarted","Data":"4ad21517fd1025e02de4ee74cdab9835142f760085b1d7e3f61a957d3f30944d"}
Sep 11 13:22:05.130077 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:05.129892    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:05.158954 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:05.158905    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt" podStartSLOduration=1.696912337 podStartE2EDuration="2.158888881s" totalImagesPullingTime="461.976544ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:22:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:05.15714285 +0000 UTC m=+1048.474599891" watchObservedRunningTime="2026-09-11 13:22:05.158888881 +0000 UTC m=+1048.476345921"
Sep 11 13:22:05.192957 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:05.192920    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:06.501495 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:06.501467    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:07.533833 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:07.533802    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:08.210303 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.210267    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp" podUID="510ce1f2-803a-433e-930d-b6af6e59ff22" containerName="main" containerID="cri-o://600fb5cc24816af6597baa2ec53911ac169d0263cfc63b7e4bfae34dfb76d947" gracePeriod=45
Sep 11 13:22:08.438517 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.438487    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:08.472152 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.472101    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:22:08.596263 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.596234    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-model-cache\") pod \"510ce1f2-803a-433e-930d-b6af6e59ff22\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") "
Sep 11 13:22:08.596733 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.596272    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-tmp-dir\") pod \"510ce1f2-803a-433e-930d-b6af6e59ff22\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") "
Sep 11 13:22:08.596733 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.596289    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-dshm\") pod \"510ce1f2-803a-433e-930d-b6af6e59ff22\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") "
Sep 11 13:22:08.596733 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.596304    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-home\") pod \"510ce1f2-803a-433e-930d-b6af6e59ff22\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") "
Sep 11 13:22:08.596733 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.596483    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/510ce1f2-803a-433e-930d-b6af6e59ff22-tls-certs\") pod \"510ce1f2-803a-433e-930d-b6af6e59ff22\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") "
Sep 11 13:22:08.596733 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.596534    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v59r9\" (UniqueName: \"kubernetes.io/projected/510ce1f2-803a-433e-930d-b6af6e59ff22-kube-api-access-v59r9\") pod \"510ce1f2-803a-433e-930d-b6af6e59ff22\" (UID: \"510ce1f2-803a-433e-930d-b6af6e59ff22\") "
Sep 11 13:22:08.596733 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.596567    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-home" (OuterVolumeSpecName: "home") pod "510ce1f2-803a-433e-930d-b6af6e59ff22" (UID: "510ce1f2-803a-433e-930d-b6af6e59ff22"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:08.596733 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.596582    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-model-cache" (OuterVolumeSpecName: "model-cache") pod "510ce1f2-803a-433e-930d-b6af6e59ff22" (UID: "510ce1f2-803a-433e-930d-b6af6e59ff22"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:08.596733 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.596594    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "510ce1f2-803a-433e-930d-b6af6e59ff22" (UID: "510ce1f2-803a-433e-930d-b6af6e59ff22"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:08.597117 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.596830    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:08.597117 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.596850    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:08.597117 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.596863    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:08.598593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.598540    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-dshm" (OuterVolumeSpecName: "dshm") pod "510ce1f2-803a-433e-930d-b6af6e59ff22" (UID: "510ce1f2-803a-433e-930d-b6af6e59ff22"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:08.598721 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.598669    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/510ce1f2-803a-433e-930d-b6af6e59ff22-kube-api-access-v59r9" (OuterVolumeSpecName: "kube-api-access-v59r9") pod "510ce1f2-803a-433e-930d-b6af6e59ff22" (UID: "510ce1f2-803a-433e-930d-b6af6e59ff22"). InnerVolumeSpecName "kube-api-access-v59r9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:22:08.598721 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.598653    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/510ce1f2-803a-433e-930d-b6af6e59ff22-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "510ce1f2-803a-433e-930d-b6af6e59ff22" (UID: "510ce1f2-803a-433e-930d-b6af6e59ff22"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:22:08.697260 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.697224    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/510ce1f2-803a-433e-930d-b6af6e59ff22-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:08.697260 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.697256    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v59r9\" (UniqueName: \"kubernetes.io/projected/510ce1f2-803a-433e-930d-b6af6e59ff22-kube-api-access-v59r9\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:08.697260 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:08.697266    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/510ce1f2-803a-433e-930d-b6af6e59ff22-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:09.154304 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:09.154270    2554 generic.go:356] "Generic (PLEG): container finished" podID="510ce1f2-803a-433e-930d-b6af6e59ff22" containerID="600fb5cc24816af6597baa2ec53911ac169d0263cfc63b7e4bfae34dfb76d947" exitCode=0
Sep 11 13:22:09.154304 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:09.154305    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"
Sep 11 13:22:09.154571 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:09.154346    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp" event={"ID":"510ce1f2-803a-433e-930d-b6af6e59ff22","Type":"ContainerDied","Data":"600fb5cc24816af6597baa2ec53911ac169d0263cfc63b7e4bfae34dfb76d947"}
Sep 11 13:22:09.154571 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:09.154493    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp" event={"ID":"510ce1f2-803a-433e-930d-b6af6e59ff22","Type":"ContainerDied","Data":"6cf101a962aa5d321a8b4e185b6c3b9ae8fb4b4dff17faa43302ff7427bcf1ad"}
Sep 11 13:22:09.154571 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:09.154520    2554 scope.go:117] "RemoveContainer" containerID="600fb5cc24816af6597baa2ec53911ac169d0263cfc63b7e4bfae34dfb76d947"
Sep 11 13:22:09.163825 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:09.163805    2554 scope.go:117] "RemoveContainer" containerID="600fb5cc24816af6597baa2ec53911ac169d0263cfc63b7e4bfae34dfb76d947"
Sep 11 13:22:09.164282 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:22:09.164258    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"600fb5cc24816af6597baa2ec53911ac169d0263cfc63b7e4bfae34dfb76d947\": container with ID starting with 600fb5cc24816af6597baa2ec53911ac169d0263cfc63b7e4bfae34dfb76d947 not found: ID does not exist" containerID="600fb5cc24816af6597baa2ec53911ac169d0263cfc63b7e4bfae34dfb76d947"
Sep 11 13:22:09.164358 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:09.164296    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"600fb5cc24816af6597baa2ec53911ac169d0263cfc63b7e4bfae34dfb76d947"} err="failed to get container status \"600fb5cc24816af6597baa2ec53911ac169d0263cfc63b7e4bfae34dfb76d947\": rpc error: code = NotFound desc = could not find container \"600fb5cc24816af6597baa2ec53911ac169d0263cfc63b7e4bfae34dfb76d947\": container with ID starting with 600fb5cc24816af6597baa2ec53911ac169d0263cfc63b7e4bfae34dfb76d947 not found: ID does not exist"
Sep 11 13:22:09.179313 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:09.179288    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"]
Sep 11 13:22:09.183730 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:09.183709    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-783a5c5b-kserve-bdcf6f575-vbwfp"]
Sep 11 13:22:09.354277 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:09.354246    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:10.140679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:10.140640    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:11.140773 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:11.140735    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:22:11.153494 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:11.153462    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="510ce1f2-803a-433e-930d-b6af6e59ff22" path="/var/lib/kubelet/pods/510ce1f2-803a-433e-930d-b6af6e59ff22/volumes"
Sep 11 13:22:11.277282 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:11.277252    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:12.572169 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:12.572138    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:17.660871 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.660830    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"]
Sep 11 13:22:17.661274 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.661099    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt" podUID="f6f30b52-e5d5-41f9-abd0-affd163c68f9" containerName="maas-api" containerID="cri-o://7070231e47f8a1c25c349d979d996b3bfa79c6b02537337c5550462b6fc89643" gracePeriod=30
Sep 11 13:22:17.693074 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.693039    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:17.821219 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.821188    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:17.910098 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.910074    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:22:17.934404 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.934305    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:17.974887 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.974860    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f30b52-e5d5-41f9-abd0-affd163c68f9-metrics-certs\") pod \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") "
Sep 11 13:22:17.974887 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.974898    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6f30b52-e5d5-41f9-abd0-affd163c68f9-openshift-service-ca\") pod \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") "
Sep 11 13:22:17.975137 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.974939    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6f30b52-e5d5-41f9-abd0-affd163c68f9-maas-api-tls\") pod \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") "
Sep 11 13:22:17.975137 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.974963    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59jrk\" (UniqueName: \"kubernetes.io/projected/f6f30b52-e5d5-41f9-abd0-affd163c68f9-kube-api-access-59jrk\") pod \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\" (UID: \"f6f30b52-e5d5-41f9-abd0-affd163c68f9\") "
Sep 11 13:22:17.975448 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.975419    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6f30b52-e5d5-41f9-abd0-affd163c68f9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f6f30b52-e5d5-41f9-abd0-affd163c68f9" (UID: "f6f30b52-e5d5-41f9-abd0-affd163c68f9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:22:17.978113 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.978076    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6f30b52-e5d5-41f9-abd0-affd163c68f9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f6f30b52-e5d5-41f9-abd0-affd163c68f9" (UID: "f6f30b52-e5d5-41f9-abd0-affd163c68f9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:22:17.978113 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.978086    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6f30b52-e5d5-41f9-abd0-affd163c68f9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f6f30b52-e5d5-41f9-abd0-affd163c68f9" (UID: "f6f30b52-e5d5-41f9-abd0-affd163c68f9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:22:17.978273 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:17.978142    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6f30b52-e5d5-41f9-abd0-affd163c68f9-kube-api-access-59jrk" (OuterVolumeSpecName: "kube-api-access-59jrk") pod "f6f30b52-e5d5-41f9-abd0-affd163c68f9" (UID: "f6f30b52-e5d5-41f9-abd0-affd163c68f9"). InnerVolumeSpecName "kube-api-access-59jrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:22:18.075925 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.075873    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6f30b52-e5d5-41f9-abd0-affd163c68f9-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:18.075925 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.075925    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59jrk\" (UniqueName: \"kubernetes.io/projected/f6f30b52-e5d5-41f9-abd0-affd163c68f9-kube-api-access-59jrk\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:18.076128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.075941    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f30b52-e5d5-41f9-abd0-affd163c68f9-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:18.076128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.075956    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6f30b52-e5d5-41f9-abd0-affd163c68f9-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:18.165167 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.165129    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:18.202019 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.201935    2554 generic.go:356] "Generic (PLEG): container finished" podID="f6f30b52-e5d5-41f9-abd0-affd163c68f9" containerID="7070231e47f8a1c25c349d979d996b3bfa79c6b02537337c5550462b6fc89643" exitCode=0
Sep 11 13:22:18.202019 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.201976    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"
Sep 11 13:22:18.202019 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.201994    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt" event={"ID":"f6f30b52-e5d5-41f9-abd0-affd163c68f9","Type":"ContainerDied","Data":"7070231e47f8a1c25c349d979d996b3bfa79c6b02537337c5550462b6fc89643"}
Sep 11 13:22:18.202263 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.202035    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt" event={"ID":"f6f30b52-e5d5-41f9-abd0-affd163c68f9","Type":"ContainerDied","Data":"f9e1b79330d75c39bf5d6072ffa7a21a63c0c872e18ed5341c6e95a3e835e6cd"}
Sep 11 13:22:18.202263 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.202055    2554 scope.go:117] "RemoveContainer" containerID="7070231e47f8a1c25c349d979d996b3bfa79c6b02537337c5550462b6fc89643"
Sep 11 13:22:18.211553 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.211529    2554 scope.go:117] "RemoveContainer" containerID="7070231e47f8a1c25c349d979d996b3bfa79c6b02537337c5550462b6fc89643"
Sep 11 13:22:18.211895 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:22:18.211876    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7070231e47f8a1c25c349d979d996b3bfa79c6b02537337c5550462b6fc89643\": container with ID starting with 7070231e47f8a1c25c349d979d996b3bfa79c6b02537337c5550462b6fc89643 not found: ID does not exist" containerID="7070231e47f8a1c25c349d979d996b3bfa79c6b02537337c5550462b6fc89643"
Sep 11 13:22:18.211990 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.211901    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7070231e47f8a1c25c349d979d996b3bfa79c6b02537337c5550462b6fc89643"} err="failed to get container status \"7070231e47f8a1c25c349d979d996b3bfa79c6b02537337c5550462b6fc89643\": rpc error: code = NotFound desc = could not find container \"7070231e47f8a1c25c349d979d996b3bfa79c6b02537337c5550462b6fc89643\": container with ID starting with 7070231e47f8a1c25c349d979d996b3bfa79c6b02537337c5550462b6fc89643 not found: ID does not exist"
Sep 11 13:22:18.230649 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.230616    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"]
Sep 11 13:22:18.238711 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.238680    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-adc38cd1-d7f84d789-4vjdt"]
Sep 11 13:22:18.266346 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.266314    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:18.659118 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.659081    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:18.966537 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:18.966451    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:19.094993 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:19.094956    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:19.154239 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:19.154207    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6f30b52-e5d5-41f9-abd0-affd163c68f9" path="/var/lib/kubelet/pods/f6f30b52-e5d5-41f9-abd0-affd163c68f9/volumes"
Sep 11 13:22:19.990581 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:19.990552    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:20.612510 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:20.612480    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:20.851764 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:20.851731    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:21.857592 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:21.857553    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:22.669145 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:22.669113    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:23.751088 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:23.751051    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:24.679422 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:24.679386    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:24.861674 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:24.861635    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:26.009331 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:26.009302    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:27.062657 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:27.062620    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:28.226344 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:28.226307    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:28.349545 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:28.349509    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:28.439491 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:28.439455    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:28.542349 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:28.542256    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:28.998666 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:28.998631    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:30.034147 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:30.034060    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:32.432165 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:32.432134    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"]
Sep 11 13:22:32.456644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:32.456611    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:40.399845 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:40.399808    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:40.463140 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:40.463103    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:42.998424 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:42.998389    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"]
Sep 11 13:22:42.998903 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:42.998880    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="510ce1f2-803a-433e-930d-b6af6e59ff22" containerName="main"
Sep 11 13:22:42.998903 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:42.998896    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="510ce1f2-803a-433e-930d-b6af6e59ff22" containerName="main"
Sep 11 13:22:42.999029 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:42.998932    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6f30b52-e5d5-41f9-abd0-affd163c68f9" containerName="maas-api"
Sep 11 13:22:42.999029 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:42.998942    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6f30b52-e5d5-41f9-abd0-affd163c68f9" containerName="maas-api"
Sep 11 13:22:42.999029 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:42.999021    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6f30b52-e5d5-41f9-abd0-affd163c68f9" containerName="maas-api"
Sep 11 13:22:42.999213 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:42.999037    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="510ce1f2-803a-433e-930d-b6af6e59ff22" containerName="main"
Sep 11 13:22:43.003566 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.003542    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.006117 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.006094    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-1c07a9\"/\"openshift-service-ca.crt\""
Sep 11 13:22:43.006835 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.006813    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-1c07a9\"/\"kube-root-ca.crt\""
Sep 11 13:22:43.006935 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.006818    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-1c07a9\"/\"explicit-model-1c07a9-kserve-self-signed-certs\""
Sep 11 13:22:43.006935 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.006821    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-1c07a9\"/\"default-dockercfg-zfkbj\""
Sep 11 13:22:43.012553 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.012524    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:43.012727 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.012707    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"]
Sep 11 13:22:43.096379 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.096326    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-model-cache\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.096576 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.096405    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-dshm\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.096576 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.096434    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-tmp-dir\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.096576 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.096544    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-tls-certs\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.096744 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.096582    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kfq6\" (UniqueName: \"kubernetes.io/projected/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-kube-api-access-4kfq6\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.096744 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.096632    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-home\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.197242 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.197208    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-tmp-dir\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.197435 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.197352    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-tls-certs\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.197435 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.197403    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kfq6\" (UniqueName: \"kubernetes.io/projected/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-kube-api-access-4kfq6\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.197561 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.197439    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-home\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.197680 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.197651    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-tmp-dir\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.197834 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.197816    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-home\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.197893 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.197689    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-model-cache\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.197893 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.197883    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-dshm\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.197984 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.197944    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-model-cache\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.199837 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.199808    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-dshm\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.199944 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.199910    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-tls-certs\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.207553 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.207499    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kfq6\" (UniqueName: \"kubernetes.io/projected/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-kube-api-access-4kfq6\") pod \"explicit-model-1c07a9-kserve-db9d6455-tklds\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") " pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.314470 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.314320    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:43.593229 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:43.593191    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"]
Sep 11 13:22:43.597170 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:22:43.597146    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f905f34_cfc5_4ae7_8d32_0fef26d42fba.slice/crio-2332c5e5bbc66c78d14ad032d802c3f8f09fe98c6b56bf71b256909045077ed7 WatchSource:0}: Error finding container 2332c5e5bbc66c78d14ad032d802c3f8f09fe98c6b56bf71b256909045077ed7: Status 404 returned error can't find the container with id 2332c5e5bbc66c78d14ad032d802c3f8f09fe98c6b56bf71b256909045077ed7
Sep 11 13:22:44.337658 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:44.337615    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds" event={"ID":"1f905f34-cfc5-4ae7-8d32-0fef26d42fba","Type":"ContainerStarted","Data":"15b06a3f3ccad3e61396c20f66d29506ed5863662aec07c63df17417ec6af789"}
Sep 11 13:22:44.337658 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:44.337666    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds" event={"ID":"1f905f34-cfc5-4ae7-8d32-0fef26d42fba","Type":"ContainerStarted","Data":"2332c5e5bbc66c78d14ad032d802c3f8f09fe98c6b56bf71b256909045077ed7"}
Sep 11 13:22:44.355612 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:44.355566    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds" podStartSLOduration=2.355548576 podStartE2EDuration="2.355548576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:22:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:44.355003991 +0000 UTC m=+1087.672461032" watchObservedRunningTime="2026-09-11 13:22:44.355548576 +0000 UTC m=+1087.673005648"
Sep 11 13:22:47.451259 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.451219    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54" podUID="009d045f-1fb1-4291-a7e6-8839ab8fe4b8" containerName="main" containerID="cri-o://2a3a2277a7adc751adfd7195fedb58005ee3380a185e141abe60c1f46df7ebcc" gracePeriod=45
Sep 11 13:22:47.702424 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.702352    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:22:47.836145 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.836107    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-dshm\") pod \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") "
Sep 11 13:22:47.836145 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.836154    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-home\") pod \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") "
Sep 11 13:22:47.836390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.836183    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-tls-certs\") pod \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") "
Sep 11 13:22:47.836390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.836241    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-tmp-dir\") pod \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") "
Sep 11 13:22:47.836390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.836277    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-model-cache\") pod \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") "
Sep 11 13:22:47.836556 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.836429    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-home" (OuterVolumeSpecName: "home") pod "009d045f-1fb1-4291-a7e6-8839ab8fe4b8" (UID: "009d045f-1fb1-4291-a7e6-8839ab8fe4b8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:47.836556 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.836444    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmvqp\" (UniqueName: \"kubernetes.io/projected/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-kube-api-access-rmvqp\") pod \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\" (UID: \"009d045f-1fb1-4291-a7e6-8839ab8fe4b8\") "
Sep 11 13:22:47.836673 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.836571    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-model-cache" (OuterVolumeSpecName: "model-cache") pod "009d045f-1fb1-4291-a7e6-8839ab8fe4b8" (UID: "009d045f-1fb1-4291-a7e6-8839ab8fe4b8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:47.836673 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.836583    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "009d045f-1fb1-4291-a7e6-8839ab8fe4b8" (UID: "009d045f-1fb1-4291-a7e6-8839ab8fe4b8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:47.836783 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.836771    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:47.836859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.836790    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:47.836859 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.836820    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:47.838296 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.838269    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-dshm" (OuterVolumeSpecName: "dshm") pod "009d045f-1fb1-4291-a7e6-8839ab8fe4b8" (UID: "009d045f-1fb1-4291-a7e6-8839ab8fe4b8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:47.838442 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.838373    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "009d045f-1fb1-4291-a7e6-8839ab8fe4b8" (UID: "009d045f-1fb1-4291-a7e6-8839ab8fe4b8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:22:47.838442 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.838395    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-kube-api-access-rmvqp" (OuterVolumeSpecName: "kube-api-access-rmvqp") pod "009d045f-1fb1-4291-a7e6-8839ab8fe4b8" (UID: "009d045f-1fb1-4291-a7e6-8839ab8fe4b8"). InnerVolumeSpecName "kube-api-access-rmvqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:22:47.937639 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.937592    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:47.937639 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.937632    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:47.937639 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:47.937642    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmvqp\" (UniqueName: \"kubernetes.io/projected/009d045f-1fb1-4291-a7e6-8839ab8fe4b8-kube-api-access-rmvqp\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:22:48.359540 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:48.359506    2554 generic.go:356] "Generic (PLEG): container finished" podID="009d045f-1fb1-4291-a7e6-8839ab8fe4b8" containerID="2a3a2277a7adc751adfd7195fedb58005ee3380a185e141abe60c1f46df7ebcc" exitCode=0
Sep 11 13:22:48.359695 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:48.359550    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"
Sep 11 13:22:48.359695 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:48.359587    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54" event={"ID":"009d045f-1fb1-4291-a7e6-8839ab8fe4b8","Type":"ContainerDied","Data":"2a3a2277a7adc751adfd7195fedb58005ee3380a185e141abe60c1f46df7ebcc"}
Sep 11 13:22:48.359695 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:48.359630    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54" event={"ID":"009d045f-1fb1-4291-a7e6-8839ab8fe4b8","Type":"ContainerDied","Data":"4b4cc9748b07792852de4da3310ae4e1b96ef54a73fc7a819bfedd36170a8d0e"}
Sep 11 13:22:48.359695 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:48.359651    2554 scope.go:117] "RemoveContainer" containerID="2a3a2277a7adc751adfd7195fedb58005ee3380a185e141abe60c1f46df7ebcc"
Sep 11 13:22:48.368567 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:48.368550    2554 scope.go:117] "RemoveContainer" containerID="2a3a2277a7adc751adfd7195fedb58005ee3380a185e141abe60c1f46df7ebcc"
Sep 11 13:22:48.368852 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:22:48.368833    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a3a2277a7adc751adfd7195fedb58005ee3380a185e141abe60c1f46df7ebcc\": container with ID starting with 2a3a2277a7adc751adfd7195fedb58005ee3380a185e141abe60c1f46df7ebcc not found: ID does not exist" containerID="2a3a2277a7adc751adfd7195fedb58005ee3380a185e141abe60c1f46df7ebcc"
Sep 11 13:22:48.368894 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:48.368868    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3a2277a7adc751adfd7195fedb58005ee3380a185e141abe60c1f46df7ebcc"} err="failed to get container status \"2a3a2277a7adc751adfd7195fedb58005ee3380a185e141abe60c1f46df7ebcc\": rpc error: code = NotFound desc = could not find container \"2a3a2277a7adc751adfd7195fedb58005ee3380a185e141abe60c1f46df7ebcc\": container with ID starting with 2a3a2277a7adc751adfd7195fedb58005ee3380a185e141abe60c1f46df7ebcc not found: ID does not exist"
Sep 11 13:22:48.384374 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:48.384341    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"]
Sep 11 13:22:48.389106 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:48.389083    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-783a5c5b-kserve-847df99c89-pqc54"]
Sep 11 13:22:49.153502 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:49.153474    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="009d045f-1fb1-4291-a7e6-8839ab8fe4b8" path="/var/lib/kubelet/pods/009d045f-1fb1-4291-a7e6-8839ab8fe4b8/volumes"
Sep 11 13:22:53.314553 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:53.314515    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:53.315025 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:53.314772    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:53.327070 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:53.327043    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:53.401483 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:53.401454    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:22:54.911503 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:54.911471    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:59.136342 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:59.136310    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"]
Sep 11 13:22:59.186027 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:22:59.185989    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:00.481529 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:00.481489    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:01.378108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:01.378077    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:02.503359 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:02.503328    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:03.005041 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:03.005003    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:03.116157 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:03.116123    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:03.211011 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:03.210976    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:03.336664 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:03.336618    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:10.155758 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.155728    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"]
Sep 11 13:23:10.156155 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.155914    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt" podUID="700c665f-9faf-424c-b571-24b021cf4554" containerName="maas-api" containerID="cri-o://a0d5069dcdae2cc1e7317084a423e5810dc516fdad11aec68243037d70c8f36d" gracePeriod=30
Sep 11 13:23:10.188636 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.188601    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:10.285165 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.285137    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:10.400642 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.400589    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:10.407602 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.407550    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:23:10.438471 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.438437    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/700c665f-9faf-424c-b571-24b021cf4554-maas-api-tls\") pod \"700c665f-9faf-424c-b571-24b021cf4554\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") "
Sep 11 13:23:10.438650 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.438544    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/700c665f-9faf-424c-b571-24b021cf4554-openshift-service-ca\") pod \"700c665f-9faf-424c-b571-24b021cf4554\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") "
Sep 11 13:23:10.439018 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.438991    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whxqx\" (UniqueName: \"kubernetes.io/projected/700c665f-9faf-424c-b571-24b021cf4554-kube-api-access-whxqx\") pod \"700c665f-9faf-424c-b571-24b021cf4554\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") "
Sep 11 13:23:10.439197 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.439147    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/700c665f-9faf-424c-b571-24b021cf4554-metrics-certs\") pod \"700c665f-9faf-424c-b571-24b021cf4554\" (UID: \"700c665f-9faf-424c-b571-24b021cf4554\") "
Sep 11 13:23:10.439197 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.439016    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/700c665f-9faf-424c-b571-24b021cf4554-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "700c665f-9faf-424c-b571-24b021cf4554" (UID: "700c665f-9faf-424c-b571-24b021cf4554"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:23:10.439502 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.439481    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/700c665f-9faf-424c-b571-24b021cf4554-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:10.441923 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.441882    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/700c665f-9faf-424c-b571-24b021cf4554-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "700c665f-9faf-424c-b571-24b021cf4554" (UID: "700c665f-9faf-424c-b571-24b021cf4554"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:23:10.441923 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.441905    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/700c665f-9faf-424c-b571-24b021cf4554-kube-api-access-whxqx" (OuterVolumeSpecName: "kube-api-access-whxqx") pod "700c665f-9faf-424c-b571-24b021cf4554" (UID: "700c665f-9faf-424c-b571-24b021cf4554"). InnerVolumeSpecName "kube-api-access-whxqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:23:10.442113 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.442033    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/700c665f-9faf-424c-b571-24b021cf4554-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "700c665f-9faf-424c-b571-24b021cf4554" (UID: "700c665f-9faf-424c-b571-24b021cf4554"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:23:10.481410 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.481347    2554 generic.go:356] "Generic (PLEG): container finished" podID="700c665f-9faf-424c-b571-24b021cf4554" containerID="a0d5069dcdae2cc1e7317084a423e5810dc516fdad11aec68243037d70c8f36d" exitCode=0
Sep 11 13:23:10.481604 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.481411    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"
Sep 11 13:23:10.481604 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.481541    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt" event={"ID":"700c665f-9faf-424c-b571-24b021cf4554","Type":"ContainerDied","Data":"a0d5069dcdae2cc1e7317084a423e5810dc516fdad11aec68243037d70c8f36d"}
Sep 11 13:23:10.481716 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.481621    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt" event={"ID":"700c665f-9faf-424c-b571-24b021cf4554","Type":"ContainerDied","Data":"4ad21517fd1025e02de4ee74cdab9835142f760085b1d7e3f61a957d3f30944d"}
Sep 11 13:23:10.481716 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.481660    2554 scope.go:117] "RemoveContainer" containerID="a0d5069dcdae2cc1e7317084a423e5810dc516fdad11aec68243037d70c8f36d"
Sep 11 13:23:10.492997 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.492979    2554 scope.go:117] "RemoveContainer" containerID="a0d5069dcdae2cc1e7317084a423e5810dc516fdad11aec68243037d70c8f36d"
Sep 11 13:23:10.494413 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:23:10.494388    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0d5069dcdae2cc1e7317084a423e5810dc516fdad11aec68243037d70c8f36d\": container with ID starting with a0d5069dcdae2cc1e7317084a423e5810dc516fdad11aec68243037d70c8f36d not found: ID does not exist" containerID="a0d5069dcdae2cc1e7317084a423e5810dc516fdad11aec68243037d70c8f36d"
Sep 11 13:23:10.494501 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.494420    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0d5069dcdae2cc1e7317084a423e5810dc516fdad11aec68243037d70c8f36d"} err="failed to get container status \"a0d5069dcdae2cc1e7317084a423e5810dc516fdad11aec68243037d70c8f36d\": rpc error: code = NotFound desc = could not find container \"a0d5069dcdae2cc1e7317084a423e5810dc516fdad11aec68243037d70c8f36d\": container with ID starting with a0d5069dcdae2cc1e7317084a423e5810dc516fdad11aec68243037d70c8f36d not found: ID does not exist"
Sep 11 13:23:10.503301 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.503282    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:10.523262 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.523240    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"]
Sep 11 13:23:10.534109 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.534084    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-1c07a9-56bd6b458d-sw4wt"]
Sep 11 13:23:10.540427 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.540403    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whxqx\" (UniqueName: \"kubernetes.io/projected/700c665f-9faf-424c-b571-24b021cf4554-kube-api-access-whxqx\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:10.540427 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.540425    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/700c665f-9faf-424c-b571-24b021cf4554-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:10.540604 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.540435    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/700c665f-9faf-424c-b571-24b021cf4554-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:10.639909 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.639870    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:10.932040 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:10.932002    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:11.084622 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:11.084590    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:11.153012 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:11.152979    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="700c665f-9faf-424c-b571-24b021cf4554" path="/var/lib/kubelet/pods/700c665f-9faf-424c-b571-24b021cf4554/volumes"
Sep 11 13:23:12.046616 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:12.046582    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:13.276173 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:13.276133    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:14.155371 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.155322    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds" podUID="1f905f34-cfc5-4ae7-8d32-0fef26d42fba" containerName="main" containerID="cri-o://15b06a3f3ccad3e61396c20f66d29506ed5863662aec07c63df17417ec6af789" gracePeriod=45
Sep 11 13:23:14.410765 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.410706    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:23:14.475242 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.475212    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-model-cache\") pod \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") "
Sep 11 13:23:14.475242 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.475245    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-tmp-dir\") pod \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") "
Sep 11 13:23:14.475491 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.475308    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-dshm\") pod \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") "
Sep 11 13:23:14.475491 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.475328    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-tls-certs\") pod \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") "
Sep 11 13:23:14.475491 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.475345    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-home\") pod \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") "
Sep 11 13:23:14.475491 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.475409    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kfq6\" (UniqueName: \"kubernetes.io/projected/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-kube-api-access-4kfq6\") pod \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\" (UID: \"1f905f34-cfc5-4ae7-8d32-0fef26d42fba\") "
Sep 11 13:23:14.475737 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.475624    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-model-cache" (OuterVolumeSpecName: "model-cache") pod "1f905f34-cfc5-4ae7-8d32-0fef26d42fba" (UID: "1f905f34-cfc5-4ae7-8d32-0fef26d42fba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:14.475737 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.475693    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-home" (OuterVolumeSpecName: "home") pod "1f905f34-cfc5-4ae7-8d32-0fef26d42fba" (UID: "1f905f34-cfc5-4ae7-8d32-0fef26d42fba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:14.475862 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.475834    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1f905f34-cfc5-4ae7-8d32-0fef26d42fba" (UID: "1f905f34-cfc5-4ae7-8d32-0fef26d42fba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:14.477709 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.477685    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-kube-api-access-4kfq6" (OuterVolumeSpecName: "kube-api-access-4kfq6") pod "1f905f34-cfc5-4ae7-8d32-0fef26d42fba" (UID: "1f905f34-cfc5-4ae7-8d32-0fef26d42fba"). InnerVolumeSpecName "kube-api-access-4kfq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:23:14.477832 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.477712    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1f905f34-cfc5-4ae7-8d32-0fef26d42fba" (UID: "1f905f34-cfc5-4ae7-8d32-0fef26d42fba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:23:14.477832 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.477765    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-dshm" (OuterVolumeSpecName: "dshm") pod "1f905f34-cfc5-4ae7-8d32-0fef26d42fba" (UID: "1f905f34-cfc5-4ae7-8d32-0fef26d42fba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:14.504623 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.504592    2554 generic.go:356] "Generic (PLEG): container finished" podID="1f905f34-cfc5-4ae7-8d32-0fef26d42fba" containerID="15b06a3f3ccad3e61396c20f66d29506ed5863662aec07c63df17417ec6af789" exitCode=0
Sep 11 13:23:14.504785 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.504649    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"
Sep 11 13:23:14.504785 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.504681    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds" event={"ID":"1f905f34-cfc5-4ae7-8d32-0fef26d42fba","Type":"ContainerDied","Data":"15b06a3f3ccad3e61396c20f66d29506ed5863662aec07c63df17417ec6af789"}
Sep 11 13:23:14.504785 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.504726    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds" event={"ID":"1f905f34-cfc5-4ae7-8d32-0fef26d42fba","Type":"ContainerDied","Data":"2332c5e5bbc66c78d14ad032d802c3f8f09fe98c6b56bf71b256909045077ed7"}
Sep 11 13:23:14.504785 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.504745    2554 scope.go:117] "RemoveContainer" containerID="15b06a3f3ccad3e61396c20f66d29506ed5863662aec07c63df17417ec6af789"
Sep 11 13:23:14.513682 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.513663    2554 scope.go:117] "RemoveContainer" containerID="15b06a3f3ccad3e61396c20f66d29506ed5863662aec07c63df17417ec6af789"
Sep 11 13:23:14.513953 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:23:14.513934    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15b06a3f3ccad3e61396c20f66d29506ed5863662aec07c63df17417ec6af789\": container with ID starting with 15b06a3f3ccad3e61396c20f66d29506ed5863662aec07c63df17417ec6af789 not found: ID does not exist" containerID="15b06a3f3ccad3e61396c20f66d29506ed5863662aec07c63df17417ec6af789"
Sep 11 13:23:14.514000 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.513962    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15b06a3f3ccad3e61396c20f66d29506ed5863662aec07c63df17417ec6af789"} err="failed to get container status \"15b06a3f3ccad3e61396c20f66d29506ed5863662aec07c63df17417ec6af789\": rpc error: code = NotFound desc = could not find container \"15b06a3f3ccad3e61396c20f66d29506ed5863662aec07c63df17417ec6af789\": container with ID starting with 15b06a3f3ccad3e61396c20f66d29506ed5863662aec07c63df17417ec6af789 not found: ID does not exist"
Sep 11 13:23:14.548631 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.548600    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"]
Sep 11 13:23:14.559182 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.559158    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-1c07a9/explicit-model-1c07a9-kserve-db9d6455-tklds"]
Sep 11 13:23:14.577022 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.576995    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kfq6\" (UniqueName: \"kubernetes.io/projected/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-kube-api-access-4kfq6\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:14.577022 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.577021    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:14.577161 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.577036    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:14.577161 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.577047    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:14.577161 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.577058    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:14.577161 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:14.577068    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f905f34-cfc5-4ae7-8d32-0fef26d42fba-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:15.154115 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:15.154084    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f905f34-cfc5-4ae7-8d32-0fef26d42fba" path="/var/lib/kubelet/pods/1f905f34-cfc5-4ae7-8d32-0fef26d42fba/volumes"
Sep 11 13:23:18.498630 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:18.498580    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:18.601919 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:18.601885    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:18.757403 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:18.757291    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:19.077107 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:19.077074    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:19.176494 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:19.176457    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:19.490864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:19.490785    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:19.766673 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:19.766571    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:20.551200 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:20.551165    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:20.655430 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:20.655397    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:20.758595 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:20.758564    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:25.818999 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:25.818965    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:25.893736 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:25.893696    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:26.186861 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:26.186771    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:26.253667 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:26.253615    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:26.398225 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:26.398189    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:27.011024 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.010987    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"]
Sep 11 13:23:27.011499 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.011322    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f905f34-cfc5-4ae7-8d32-0fef26d42fba" containerName="main"
Sep 11 13:23:27.011499 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.011334    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f905f34-cfc5-4ae7-8d32-0fef26d42fba" containerName="main"
Sep 11 13:23:27.011499 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.011352    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="009d045f-1fb1-4291-a7e6-8839ab8fe4b8" containerName="main"
Sep 11 13:23:27.011499 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.011373    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="009d045f-1fb1-4291-a7e6-8839ab8fe4b8" containerName="main"
Sep 11 13:23:27.011499 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.011404    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="700c665f-9faf-424c-b571-24b021cf4554" containerName="maas-api"
Sep 11 13:23:27.011499 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.011410    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="700c665f-9faf-424c-b571-24b021cf4554" containerName="maas-api"
Sep 11 13:23:27.011499 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.011470    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f905f34-cfc5-4ae7-8d32-0fef26d42fba" containerName="main"
Sep 11 13:23:27.011499 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.011477    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="700c665f-9faf-424c-b571-24b021cf4554" containerName="maas-api"
Sep 11 13:23:27.011499 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.011485    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="009d045f-1fb1-4291-a7e6-8839ab8fe4b8" containerName="main"
Sep 11 13:23:27.015786 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.015762    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.018142 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.018122    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-4b7627\"/\"default-dockercfg-hz46n\""
Sep 11 13:23:27.018765 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.018747    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-4b7627\"/\"kube-root-ca.crt\""
Sep 11 13:23:27.018827 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.018747    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-4b7627\"/\"openshift-service-ca.crt\""
Sep 11 13:23:27.018866 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.018771    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-4b7627\"/\"noten-model-4b7627-kserve-self-signed-certs\""
Sep 11 13:23:27.026506 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.026487    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"]
Sep 11 13:23:27.099673 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.099640    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdb5a1e5-ed5c-4972-885e-ad9342858c38-tls-certs\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.099941 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.099917    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpkfc\" (UniqueName: \"kubernetes.io/projected/cdb5a1e5-ed5c-4972-885e-ad9342858c38-kube-api-access-mpkfc\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.100135 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.100096    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-home\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.100251 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.100151    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-model-cache\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.100314 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.100243    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-dshm\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.100314 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.100291    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-tmp-dir\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.108487 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.108457    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:27.200850 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.200803    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-tmp-dir\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.201026 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.200873    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdb5a1e5-ed5c-4972-885e-ad9342858c38-tls-certs\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.201026 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.200907    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpkfc\" (UniqueName: \"kubernetes.io/projected/cdb5a1e5-ed5c-4972-885e-ad9342858c38-kube-api-access-mpkfc\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.201026 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.200937    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-home\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.201026 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.200953    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-model-cache\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.201213 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.201116    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-dshm\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.201303 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.201280    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-home\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.201347 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.201304    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-tmp-dir\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.201419 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.201349    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-model-cache\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.203184 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.203160    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-dshm\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.203416 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.203398    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdb5a1e5-ed5c-4972-885e-ad9342858c38-tls-certs\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.209883 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.209862    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpkfc\" (UniqueName: \"kubernetes.io/projected/cdb5a1e5-ed5c-4972-885e-ad9342858c38-kube-api-access-mpkfc\") pod \"noten-model-4b7627-kserve-86bc96dcd7-m4bm7\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") " pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.325078 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.325046    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:27.602873 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:27.602844    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"]
Sep 11 13:23:27.604638 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:23:27.604604    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcdb5a1e5_ed5c_4972_885e_ad9342858c38.slice/crio-3967856e825f64131b882bcf7e04da0d8619e4c1db32fdf4957f457276a76131 WatchSource:0}: Error finding container 3967856e825f64131b882bcf7e04da0d8619e4c1db32fdf4957f457276a76131: Status 404 returned error can't find the container with id 3967856e825f64131b882bcf7e04da0d8619e4c1db32fdf4957f457276a76131
Sep 11 13:23:28.579078 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:28.579040    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7" event={"ID":"cdb5a1e5-ed5c-4972-885e-ad9342858c38","Type":"ContainerStarted","Data":"d0b25f1ab1e5aefddd0b26792e51a5a9c930048b6337642a58c0cb83f9eea7bc"}
Sep 11 13:23:28.579078 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:28.579084    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7" event={"ID":"cdb5a1e5-ed5c-4972-885e-ad9342858c38","Type":"ContainerStarted","Data":"3967856e825f64131b882bcf7e04da0d8619e4c1db32fdf4957f457276a76131"}
Sep 11 13:23:28.601453 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:28.601398    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7" podStartSLOduration=2.601354991 podStartE2EDuration="2.601354991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:28.600151787 +0000 UTC m=+1131.917608827" watchObservedRunningTime="2026-09-11 13:23:28.601354991 +0000 UTC m=+1131.918812069"
Sep 11 13:23:35.952390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:35.952342    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"]
Sep 11 13:23:35.979354 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:35.979324    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:36.087633 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:36.087602    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:36.330814 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:36.330779    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:36.428100 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:36.428064    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:36.930056 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:36.930023    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:37.325605 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:37.325573    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:37.714816 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:37.714781    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:41.439470 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:41.439438    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:44.163097 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:44.163060    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:44.242737 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:44.242695    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:44.336397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:44.336352    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:44.434177 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:44.434096    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:44.542093 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:44.542062    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:45.198096 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:45.198056    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:45.397157 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:45.397114    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:46.883404 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:46.883349    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:50.971126 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:50.971086    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7" podUID="cdb5a1e5-ed5c-4972-885e-ad9342858c38" containerName="main" containerID="cri-o://d0b25f1ab1e5aefddd0b26792e51a5a9c930048b6337642a58c0cb83f9eea7bc" gracePeriod=45
Sep 11 13:23:51.219047 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.219020    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:51.293561 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.293473    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-home\") pod \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") "
Sep 11 13:23:51.293561 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.293515    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-model-cache\") pod \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") "
Sep 11 13:23:51.293561 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.293553    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-tmp-dir\") pod \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") "
Sep 11 13:23:51.293846 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.293573    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdb5a1e5-ed5c-4972-885e-ad9342858c38-tls-certs\") pod \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") "
Sep 11 13:23:51.293846 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.293608    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-dshm\") pod \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") "
Sep 11 13:23:51.293846 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.293635    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpkfc\" (UniqueName: \"kubernetes.io/projected/cdb5a1e5-ed5c-4972-885e-ad9342858c38-kube-api-access-mpkfc\") pod \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\" (UID: \"cdb5a1e5-ed5c-4972-885e-ad9342858c38\") "
Sep 11 13:23:51.293846 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.293765    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-home" (OuterVolumeSpecName: "home") pod "cdb5a1e5-ed5c-4972-885e-ad9342858c38" (UID: "cdb5a1e5-ed5c-4972-885e-ad9342858c38"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:51.294002 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.293850    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cdb5a1e5-ed5c-4972-885e-ad9342858c38" (UID: "cdb5a1e5-ed5c-4972-885e-ad9342858c38"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:51.294002 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.293898    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-model-cache" (OuterVolumeSpecName: "model-cache") pod "cdb5a1e5-ed5c-4972-885e-ad9342858c38" (UID: "cdb5a1e5-ed5c-4972-885e-ad9342858c38"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:51.295950 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.295921    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdb5a1e5-ed5c-4972-885e-ad9342858c38-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cdb5a1e5-ed5c-4972-885e-ad9342858c38" (UID: "cdb5a1e5-ed5c-4972-885e-ad9342858c38"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:23:51.295950 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.295921    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-dshm" (OuterVolumeSpecName: "dshm") pod "cdb5a1e5-ed5c-4972-885e-ad9342858c38" (UID: "cdb5a1e5-ed5c-4972-885e-ad9342858c38"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:51.296101 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.295960    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdb5a1e5-ed5c-4972-885e-ad9342858c38-kube-api-access-mpkfc" (OuterVolumeSpecName: "kube-api-access-mpkfc") pod "cdb5a1e5-ed5c-4972-885e-ad9342858c38" (UID: "cdb5a1e5-ed5c-4972-885e-ad9342858c38"). InnerVolumeSpecName "kube-api-access-mpkfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:23:51.394580 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.394542    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:51.394580 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.394576    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpkfc\" (UniqueName: \"kubernetes.io/projected/cdb5a1e5-ed5c-4972-885e-ad9342858c38-kube-api-access-mpkfc\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:51.394580 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.394587    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:51.394813 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.394595    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:51.394813 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.394627    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdb5a1e5-ed5c-4972-885e-ad9342858c38-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:51.394813 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.394635    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdb5a1e5-ed5c-4972-885e-ad9342858c38-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:51.692331 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.692281    2554 generic.go:356] "Generic (PLEG): container finished" podID="cdb5a1e5-ed5c-4972-885e-ad9342858c38" containerID="d0b25f1ab1e5aefddd0b26792e51a5a9c930048b6337642a58c0cb83f9eea7bc" exitCode=0
Sep 11 13:23:51.692331 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.692341    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"
Sep 11 13:23:51.692331 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.692325    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7" event={"ID":"cdb5a1e5-ed5c-4972-885e-ad9342858c38","Type":"ContainerDied","Data":"d0b25f1ab1e5aefddd0b26792e51a5a9c930048b6337642a58c0cb83f9eea7bc"}
Sep 11 13:23:51.692698 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.692574    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7" event={"ID":"cdb5a1e5-ed5c-4972-885e-ad9342858c38","Type":"ContainerDied","Data":"3967856e825f64131b882bcf7e04da0d8619e4c1db32fdf4957f457276a76131"}
Sep 11 13:23:51.692698 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.692607    2554 scope.go:117] "RemoveContainer" containerID="d0b25f1ab1e5aefddd0b26792e51a5a9c930048b6337642a58c0cb83f9eea7bc"
Sep 11 13:23:51.703325 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.703285    2554 scope.go:117] "RemoveContainer" containerID="d0b25f1ab1e5aefddd0b26792e51a5a9c930048b6337642a58c0cb83f9eea7bc"
Sep 11 13:23:51.703685 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:23:51.703658    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0b25f1ab1e5aefddd0b26792e51a5a9c930048b6337642a58c0cb83f9eea7bc\": container with ID starting with d0b25f1ab1e5aefddd0b26792e51a5a9c930048b6337642a58c0cb83f9eea7bc not found: ID does not exist" containerID="d0b25f1ab1e5aefddd0b26792e51a5a9c930048b6337642a58c0cb83f9eea7bc"
Sep 11 13:23:51.703772 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.703696    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0b25f1ab1e5aefddd0b26792e51a5a9c930048b6337642a58c0cb83f9eea7bc"} err="failed to get container status \"d0b25f1ab1e5aefddd0b26792e51a5a9c930048b6337642a58c0cb83f9eea7bc\": rpc error: code = NotFound desc = could not find container \"d0b25f1ab1e5aefddd0b26792e51a5a9c930048b6337642a58c0cb83f9eea7bc\": container with ID starting with d0b25f1ab1e5aefddd0b26792e51a5a9c930048b6337642a58c0cb83f9eea7bc not found: ID does not exist"
Sep 11 13:23:51.718776 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.718749    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"]
Sep 11 13:23:51.729600 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.729574    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"]
Sep 11 13:23:51.736449 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.736421    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-4b7627/noten-model-4b7627-kserve-86bc96dcd7-m4bm7"]
Sep 11 13:23:51.773385 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:51.773341    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:52.805730 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:52.805680    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:52.881124 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:52.881089    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg"]
Sep 11 13:23:52.881547 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:52.881532    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdb5a1e5-ed5c-4972-885e-ad9342858c38" containerName="main"
Sep 11 13:23:52.881547 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:52.881548    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdb5a1e5-ed5c-4972-885e-ad9342858c38" containerName="main"
Sep 11 13:23:52.881669 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:52.881612    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdb5a1e5-ed5c-4972-885e-ad9342858c38" containerName="main"
Sep 11 13:23:52.885910 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:52.885893    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg"
Sep 11 13:23:52.888691 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:52.888661    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:23:52.888691 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:52.888670    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:23:52.898380 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:52.898340    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg"]
Sep 11 13:23:52.907187 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:52.907165    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcfe30dd-98c4-454e-aaec-228e92c755d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg\" (UID: \"bcfe30dd-98c4-454e-aaec-228e92c755d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg"
Sep 11 13:23:53.008118 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:53.008079    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcfe30dd-98c4-454e-aaec-228e92c755d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg\" (UID: \"bcfe30dd-98c4-454e-aaec-228e92c755d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg"
Sep 11 13:23:53.008754 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:53.008734    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcfe30dd-98c4-454e-aaec-228e92c755d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg\" (UID: \"bcfe30dd-98c4-454e-aaec-228e92c755d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg"
Sep 11 13:23:53.153601 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:53.153562    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdb5a1e5-ed5c-4972-885e-ad9342858c38" path="/var/lib/kubelet/pods/cdb5a1e5-ed5c-4972-885e-ad9342858c38/volumes"
Sep 11 13:23:53.195647 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:53.195612    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg"
Sep 11 13:23:53.512406 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:53.512282    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg"]
Sep 11 13:23:53.514709 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:23:53.514684    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcfe30dd_98c4_454e_aaec_228e92c755d7.slice/crio-78fa6265e8fad062a4c3a6b9dde9432647deedf857274a9375b8325612fceb95 WatchSource:0}: Error finding container 78fa6265e8fad062a4c3a6b9dde9432647deedf857274a9375b8325612fceb95: Status 404 returned error can't find the container with id 78fa6265e8fad062a4c3a6b9dde9432647deedf857274a9375b8325612fceb95
Sep 11 13:23:53.704356 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:53.704320    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg" event={"ID":"bcfe30dd-98c4-454e-aaec-228e92c755d7","Type":"ContainerStarted","Data":"78fa6265e8fad062a4c3a6b9dde9432647deedf857274a9375b8325612fceb95"}
Sep 11 13:23:54.277061 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:54.277025    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:56.724556 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:56.724496    2554 generic.go:356] "Generic (PLEG): container finished" podID="bcfe30dd-98c4-454e-aaec-228e92c755d7" containerID="0dc04153248f38488cfe3f86077504b01c7aaacaa1c881ea621203741570b675" exitCode=0
Sep 11 13:23:56.725035 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:56.724561    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg" event={"ID":"bcfe30dd-98c4-454e-aaec-228e92c755d7","Type":"ContainerDied","Data":"0dc04153248f38488cfe3f86077504b01c7aaacaa1c881ea621203741570b675"}
Sep 11 13:23:56.761779 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:56.761751    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:57.558560 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:57.558529    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:57.710145 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:57.710101    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:57.860771 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:57.860740    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg"
Sep 11 13:23:57.951864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:57.951831    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcfe30dd-98c4-454e-aaec-228e92c755d7-maas-api-service-ca\") pod \"bcfe30dd-98c4-454e-aaec-228e92c755d7\" (UID: \"bcfe30dd-98c4-454e-aaec-228e92c755d7\") "
Sep 11 13:23:57.952217 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:57.952193    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcfe30dd-98c4-454e-aaec-228e92c755d7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bcfe30dd-98c4-454e-aaec-228e92c755d7" (UID: "bcfe30dd-98c4-454e-aaec-228e92c755d7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:23:58.052731 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:58.052692    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcfe30dd-98c4-454e-aaec-228e92c755d7-maas-api-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:23:58.352280 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:58.352240    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:58.613148 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:58.613059    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:58.735340 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:58.735310    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg"
Sep 11 13:23:58.735530 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:58.735312    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg" event={"ID":"bcfe30dd-98c4-454e-aaec-228e92c755d7","Type":"ContainerDied","Data":"78fa6265e8fad062a4c3a6b9dde9432647deedf857274a9375b8325612fceb95"}
Sep 11 13:23:58.735530 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:23:58.735444    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78fa6265e8fad062a4c3a6b9dde9432647deedf857274a9375b8325612fceb95"
Sep 11 13:24:04.225197 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:04.225154    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:04.574356 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:04.574320    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:04.739130 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:04.739085    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:04.865877 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:04.865798    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg"]
Sep 11 13:24:04.870058 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:04.870033    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a649dc-bz5lg"]
Sep 11 13:24:04.954316 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:04.954285    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:05.158353 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:05.158261    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcfe30dd-98c4-454e-aaec-228e92c755d7" path="/var/lib/kubelet/pods/bcfe30dd-98c4-454e-aaec-228e92c755d7/volumes"
Sep 11 13:24:05.258194 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:05.258159    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:05.578408 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:05.578356    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:06.739760 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:06.739717    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz" podUID="65646e1b-5b96-43bd-9f67-53b41bc86a95" containerName="main" containerID="cri-o://9e6c98f122924e3f6d3641019bfd28cb653db67bb5d68c9de2e9294681b18cd2" gracePeriod=45
Sep 11 13:24:06.999127 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:06.999057    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:24:07.044827 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.044791    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ms2c\" (UniqueName: \"kubernetes.io/projected/65646e1b-5b96-43bd-9f67-53b41bc86a95-kube-api-access-7ms2c\") pod \"65646e1b-5b96-43bd-9f67-53b41bc86a95\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") "
Sep 11 13:24:07.044827 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.044833    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65646e1b-5b96-43bd-9f67-53b41bc86a95-tls-certs\") pod \"65646e1b-5b96-43bd-9f67-53b41bc86a95\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") "
Sep 11 13:24:07.045064 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.044866    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-tmp-dir\") pod \"65646e1b-5b96-43bd-9f67-53b41bc86a95\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") "
Sep 11 13:24:07.045064 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.044891    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-dshm\") pod \"65646e1b-5b96-43bd-9f67-53b41bc86a95\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") "
Sep 11 13:24:07.045222 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.045194    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "65646e1b-5b96-43bd-9f67-53b41bc86a95" (UID: "65646e1b-5b96-43bd-9f67-53b41bc86a95"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:07.045286 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.045217    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-home" (OuterVolumeSpecName: "home") pod "65646e1b-5b96-43bd-9f67-53b41bc86a95" (UID: "65646e1b-5b96-43bd-9f67-53b41bc86a95"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:07.045338 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.045282    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-home\") pod \"65646e1b-5b96-43bd-9f67-53b41bc86a95\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") "
Sep 11 13:24:07.045338 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.045316    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-model-cache\") pod \"65646e1b-5b96-43bd-9f67-53b41bc86a95\" (UID: \"65646e1b-5b96-43bd-9f67-53b41bc86a95\") "
Sep 11 13:24:07.045613 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.045585    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-model-cache" (OuterVolumeSpecName: "model-cache") pod "65646e1b-5b96-43bd-9f67-53b41bc86a95" (UID: "65646e1b-5b96-43bd-9f67-53b41bc86a95"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:07.045751 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.045672    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:24:07.045751 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.045688    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:24:07.045751 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.045700    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:24:07.047435 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.047409    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65646e1b-5b96-43bd-9f67-53b41bc86a95-kube-api-access-7ms2c" (OuterVolumeSpecName: "kube-api-access-7ms2c") pod "65646e1b-5b96-43bd-9f67-53b41bc86a95" (UID: "65646e1b-5b96-43bd-9f67-53b41bc86a95"). InnerVolumeSpecName "kube-api-access-7ms2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:07.047557 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.047486    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65646e1b-5b96-43bd-9f67-53b41bc86a95-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "65646e1b-5b96-43bd-9f67-53b41bc86a95" (UID: "65646e1b-5b96-43bd-9f67-53b41bc86a95"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:07.047656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.047636    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-dshm" (OuterVolumeSpecName: "dshm") pod "65646e1b-5b96-43bd-9f67-53b41bc86a95" (UID: "65646e1b-5b96-43bd-9f67-53b41bc86a95"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:07.094430 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.094397    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:07.146313 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.146275    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ms2c\" (UniqueName: \"kubernetes.io/projected/65646e1b-5b96-43bd-9f67-53b41bc86a95-kube-api-access-7ms2c\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:24:07.146313 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.146307    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65646e1b-5b96-43bd-9f67-53b41bc86a95-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:24:07.146313 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.146320    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65646e1b-5b96-43bd-9f67-53b41bc86a95-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:24:07.792940 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.792904    2554 generic.go:356] "Generic (PLEG): container finished" podID="65646e1b-5b96-43bd-9f67-53b41bc86a95" containerID="9e6c98f122924e3f6d3641019bfd28cb653db67bb5d68c9de2e9294681b18cd2" exitCode=0
Sep 11 13:24:07.793388 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.792945    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"
Sep 11 13:24:07.793388 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.792986    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz" event={"ID":"65646e1b-5b96-43bd-9f67-53b41bc86a95","Type":"ContainerDied","Data":"9e6c98f122924e3f6d3641019bfd28cb653db67bb5d68c9de2e9294681b18cd2"}
Sep 11 13:24:07.793388 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.793022    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz" event={"ID":"65646e1b-5b96-43bd-9f67-53b41bc86a95","Type":"ContainerDied","Data":"6719d324d53baa6dd9dde6fc286e458513709508a5245b3f11adb54bdbadaa27"}
Sep 11 13:24:07.793388 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.793039    2554 scope.go:117] "RemoveContainer" containerID="9e6c98f122924e3f6d3641019bfd28cb653db67bb5d68c9de2e9294681b18cd2"
Sep 11 13:24:07.801770 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.801751    2554 scope.go:117] "RemoveContainer" containerID="9e6c98f122924e3f6d3641019bfd28cb653db67bb5d68c9de2e9294681b18cd2"
Sep 11 13:24:07.802091 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:24:07.802067    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e6c98f122924e3f6d3641019bfd28cb653db67bb5d68c9de2e9294681b18cd2\": container with ID starting with 9e6c98f122924e3f6d3641019bfd28cb653db67bb5d68c9de2e9294681b18cd2 not found: ID does not exist" containerID="9e6c98f122924e3f6d3641019bfd28cb653db67bb5d68c9de2e9294681b18cd2"
Sep 11 13:24:07.802139 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.802102    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e6c98f122924e3f6d3641019bfd28cb653db67bb5d68c9de2e9294681b18cd2"} err="failed to get container status \"9e6c98f122924e3f6d3641019bfd28cb653db67bb5d68c9de2e9294681b18cd2\": rpc error: code = NotFound desc = could not find container \"9e6c98f122924e3f6d3641019bfd28cb653db67bb5d68c9de2e9294681b18cd2\": container with ID starting with 9e6c98f122924e3f6d3641019bfd28cb653db67bb5d68c9de2e9294681b18cd2 not found: ID does not exist"
Sep 11 13:24:07.823845 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.823810    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"]
Sep 11 13:24:07.835496 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:07.835463    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a649dc/test-model-a649dc-kserve-768bc778c5-4fqhz"]
Sep 11 13:24:09.117317 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:09.117278    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:09.153118 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:09.153089    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65646e1b-5b96-43bd-9f67-53b41bc86a95" path="/var/lib/kubelet/pods/65646e1b-5b96-43bd-9f67-53b41bc86a95/volumes"
Sep 11 13:24:09.394526 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:09.394443    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:09.618629 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:09.618589    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:09.960075 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:09.960017    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:10.324882 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:10.324838    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:13.454093 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:13.454043    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:14.001461 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:14.001407    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:14.434129 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:14.434066    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:16.052517 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:16.052476    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:17.032596 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.032562    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:17.202416 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.202318    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:17.226193 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.226159    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"]
Sep 11 13:24:17.226641 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.226624    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65646e1b-5b96-43bd-9f67-53b41bc86a95" containerName="main"
Sep 11 13:24:17.226705 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.226644    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="65646e1b-5b96-43bd-9f67-53b41bc86a95" containerName="main"
Sep 11 13:24:17.226705 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.226658    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcfe30dd-98c4-454e-aaec-228e92c755d7" containerName="revoke-keys"
Sep 11 13:24:17.226705 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.226664    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcfe30dd-98c4-454e-aaec-228e92c755d7" containerName="revoke-keys"
Sep 11 13:24:17.226814 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.226722    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcfe30dd-98c4-454e-aaec-228e92c755d7" containerName="revoke-keys"
Sep 11 13:24:17.226814 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.226731    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="65646e1b-5b96-43bd-9f67-53b41bc86a95" containerName="main"
Sep 11 13:24:17.231243 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.231219    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.236581 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.236546    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f78ffd60-serving-cert\""
Sep 11 13:24:17.236752 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.236618    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:24:17.237846 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.237814    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 13:24:17.237996 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.237863    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 13:24:17.238281 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.238264    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zxpw2\""
Sep 11 13:24:17.255621 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.255573    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"]
Sep 11 13:24:17.340062 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.340025    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f4b856-1431-466f-bb4a-a490e0724623-openshift-service-ca\") pod \"maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.340266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.340223    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f4b856-1431-466f-bb4a-a490e0724623-maas-api-tls\") pod \"maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.340341 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.340308    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6t5z\" (UniqueName: \"kubernetes.io/projected/33f4b856-1431-466f-bb4a-a490e0724623-kube-api-access-t6t5z\") pod \"maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.340433 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.340417    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33f4b856-1431-466f-bb4a-a490e0724623-metrics-certs\") pod \"maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.358504 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.358463    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:17.441824 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.441785    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f4b856-1431-466f-bb4a-a490e0724623-maas-api-tls\") pod \"maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.442007 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.441836    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6t5z\" (UniqueName: \"kubernetes.io/projected/33f4b856-1431-466f-bb4a-a490e0724623-kube-api-access-t6t5z\") pod \"maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.442007 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.441893    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33f4b856-1431-466f-bb4a-a490e0724623-metrics-certs\") pod \"maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.442007 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.441927    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f4b856-1431-466f-bb4a-a490e0724623-openshift-service-ca\") pod \"maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.442671 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.442646    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f4b856-1431-466f-bb4a-a490e0724623-openshift-service-ca\") pod \"maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.444481 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.444459    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f4b856-1431-466f-bb4a-a490e0724623-maas-api-tls\") pod \"maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.444563 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.444509    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33f4b856-1431-466f-bb4a-a490e0724623-metrics-certs\") pod \"maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.477050 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.477016    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6t5z\" (UniqueName: \"kubernetes.io/projected/33f4b856-1431-466f-bb4a-a490e0724623-kube-api-access-t6t5z\") pod \"maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.543206 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.543158    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:17.552850 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.552668    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:17.849508 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:17.849476    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"]
Sep 11 13:24:17.851155 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:24:17.851125    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33f4b856_1431_466f_bb4a_a490e0724623.slice/crio-e4581da279ddcaa3877e9bfe63ff4d759860c946b67ca5e5b5eb75d5b01277fb WatchSource:0}: Error finding container e4581da279ddcaa3877e9bfe63ff4d759860c946b67ca5e5b5eb75d5b01277fb: Status 404 returned error can't find the container with id e4581da279ddcaa3877e9bfe63ff4d759860c946b67ca5e5b5eb75d5b01277fb
Sep 11 13:24:18.450087 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:18.450052    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:18.585254 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:18.585222    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:18.852393 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:18.852346    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s" event={"ID":"33f4b856-1431-466f-bb4a-a490e0724623","Type":"ContainerStarted","Data":"12aef2e41d0efe414f315619a6e3a5e236d93435b6b2493b02da32f6b6cbdf3d"}
Sep 11 13:24:18.852595 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:18.852413    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s" event={"ID":"33f4b856-1431-466f-bb4a-a490e0724623","Type":"ContainerStarted","Data":"e4581da279ddcaa3877e9bfe63ff4d759860c946b67ca5e5b5eb75d5b01277fb"}
Sep 11 13:24:18.852595 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:18.852510    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:18.876274 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:18.876212    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s" podStartSLOduration=1.356197645 podStartE2EDuration="1.876192571s" totalImagesPullingTime="519.994926ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:18.874323625 +0000 UTC m=+1182.191780680" watchObservedRunningTime="2026-09-11 13:24:18.876192571 +0000 UTC m=+1182.193649611"
Sep 11 13:24:19.171802 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:19.171706    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:20.236651 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:20.236620    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:20.951520 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:20.951485    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:21.071908 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:21.071870    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:24.863897 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:24.863862    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:27.842275 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:27.842239    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:28.506485 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:28.506454    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:28.582707 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:28.582665    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:28.675023 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:28.674990    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:28.754603 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:28.754570    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:29.084100 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:29.084062    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:29.265135 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:29.265085    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:30.050820 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:30.050786    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:31.645043 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:31.645010    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:32.439815 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:32.439771    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:33.726630 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:33.726596    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:34.352094 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:34.352064    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:34.452150 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:34.452109    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:34.773821 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:34.773736    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:37.157814 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:37.157786    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:24:37.159071 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:37.159046    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:24:38.194000 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:38.193969    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:38.389797 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:38.389762    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:38.645620 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:38.645583    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:39.105513 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:39.105481    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:39.642315 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:39.642284    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:39.917426 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:39.917335    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:42.245417 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:42.245354    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:42.788177 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:42.788140    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:42.914822 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:42.914785    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:43.670451 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:43.670385    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:44.514376 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:44.514334    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:44.695468 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:44.695434    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:45.520594 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.520559    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"]
Sep 11 13:24:45.525331 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.525295    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:45.529300 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.529270    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-6ffb6a-serving-cert\""
Sep 11 13:24:45.596501 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.596458    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:45.596700 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.596512    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g48dh\" (UniqueName: \"kubernetes.io/projected/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-kube-api-access-g48dh\") pod \"maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:45.596700 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.596594    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:45.596700 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.596621    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:45.611219 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.611176    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"]
Sep 11 13:24:45.636941 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.636903    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:45.697046 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.697005    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g48dh\" (UniqueName: \"kubernetes.io/projected/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-kube-api-access-g48dh\") pod \"maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:45.697558 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.697074    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:45.697558 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.697118    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:45.697558 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.697160    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:45.697558 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:24:45.697177    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-serving-cert: secret "maas-api-e2e-shared-a-w3-6ffb6a-serving-cert" not found
Sep 11 13:24:45.697558 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:24:45.697246    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-maas-api-tls podName:30e0f013-8b18-42f9-94fe-6cdf5cc7b71b nodeName:}" failed. No retries permitted until 2026-09-11 13:24:46.197228507 +0000 UTC m=+1209.514685544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-maas-api-tls") pod "maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x" (UID: "30e0f013-8b18-42f9-94fe-6cdf5cc7b71b") : secret "maas-api-e2e-shared-a-w3-6ffb6a-serving-cert" not found
Sep 11 13:24:45.697847 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.697827    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:45.699738 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.699714    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:45.726095 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.726056    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g48dh\" (UniqueName: \"kubernetes.io/projected/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-kube-api-access-g48dh\") pod \"maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:45.894609 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:45.894576    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:46.131834 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:46.131793    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:46.200574 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:46.200477    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:46.202897 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:46.202870    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:46.436441 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:46.436398    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:46.773869 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:46.773816    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"]
Sep 11 13:24:46.776278 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:24:46.776253    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30e0f013_8b18_42f9_94fe_6cdf5cc7b71b.slice/crio-3ef75f3ea2b5cf1c54400d752b3d2907604d5a9a764a46a88b0b8434da6704cd WatchSource:0}: Error finding container 3ef75f3ea2b5cf1c54400d752b3d2907604d5a9a764a46a88b0b8434da6704cd: Status 404 returned error can't find the container with id 3ef75f3ea2b5cf1c54400d752b3d2907604d5a9a764a46a88b0b8434da6704cd
Sep 11 13:24:46.830851 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:46.830809    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:46.992639 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:46.992604    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x" event={"ID":"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b","Type":"ContainerStarted","Data":"3ef75f3ea2b5cf1c54400d752b3d2907604d5a9a764a46a88b0b8434da6704cd"}
Sep 11 13:24:47.151946 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:47.151916    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:47.994507 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:47.994466    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:47.999321 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:47.999291    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x" event={"ID":"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b","Type":"ContainerStarted","Data":"f139e783431b1614f6a5c516d619f38ae07eb5ecdb48ddff40cde90e6d7bb1e9"}
Sep 11 13:24:47.999459 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:47.999425    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:48.031467 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:48.031405    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x" podStartSLOduration=2.707961868 podStartE2EDuration="3.031355952s" totalImagesPullingTime="323.394084ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:48.031164771 +0000 UTC m=+1211.348621812" watchObservedRunningTime="2026-09-11 13:24:48.031355952 +0000 UTC m=+1211.348812989"
Sep 11 13:24:49.837497 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:49.837460    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:52.091403 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:52.091340    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:54.010181 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:54.010149    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:24:55.831578 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:55.831542    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:57.021433 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.021393    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"]
Sep 11 13:24:57.021868 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.021602    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s" podUID="33f4b856-1431-466f-bb4a-a490e0724623" containerName="maas-api" containerID="cri-o://12aef2e41d0efe414f315619a6e3a5e236d93435b6b2493b02da32f6b6cbdf3d" gracePeriod=30
Sep 11 13:24:57.119588 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.119554    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:57.287536 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.287506    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:57.330694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.330654    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:57.402448 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.402404    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f4b856-1431-466f-bb4a-a490e0724623-maas-api-tls\") pod \"33f4b856-1431-466f-bb4a-a490e0724623\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") "
Sep 11 13:24:57.402653 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.402474    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f4b856-1431-466f-bb4a-a490e0724623-openshift-service-ca\") pod \"33f4b856-1431-466f-bb4a-a490e0724623\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") "
Sep 11 13:24:57.402653 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.402545    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6t5z\" (UniqueName: \"kubernetes.io/projected/33f4b856-1431-466f-bb4a-a490e0724623-kube-api-access-t6t5z\") pod \"33f4b856-1431-466f-bb4a-a490e0724623\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") "
Sep 11 13:24:57.402653 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.402610    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33f4b856-1431-466f-bb4a-a490e0724623-metrics-certs\") pod \"33f4b856-1431-466f-bb4a-a490e0724623\" (UID: \"33f4b856-1431-466f-bb4a-a490e0724623\") "
Sep 11 13:24:57.402904 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.402854    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33f4b856-1431-466f-bb4a-a490e0724623-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33f4b856-1431-466f-bb4a-a490e0724623" (UID: "33f4b856-1431-466f-bb4a-a490e0724623"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:57.404781 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.404742    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33f4b856-1431-466f-bb4a-a490e0724623-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33f4b856-1431-466f-bb4a-a490e0724623" (UID: "33f4b856-1431-466f-bb4a-a490e0724623"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:57.404901 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.404752    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33f4b856-1431-466f-bb4a-a490e0724623-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "33f4b856-1431-466f-bb4a-a490e0724623" (UID: "33f4b856-1431-466f-bb4a-a490e0724623"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:57.404901 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.404802    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33f4b856-1431-466f-bb4a-a490e0724623-kube-api-access-t6t5z" (OuterVolumeSpecName: "kube-api-access-t6t5z") pod "33f4b856-1431-466f-bb4a-a490e0724623" (UID: "33f4b856-1431-466f-bb4a-a490e0724623"). InnerVolumeSpecName "kube-api-access-t6t5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:57.503303 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.503265    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33f4b856-1431-466f-bb4a-a490e0724623-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:24:57.503303 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.503297    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f4b856-1431-466f-bb4a-a490e0724623-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:24:57.503303 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.503308    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6t5z\" (UniqueName: \"kubernetes.io/projected/33f4b856-1431-466f-bb4a-a490e0724623-kube-api-access-t6t5z\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:24:57.503581 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.503318    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33f4b856-1431-466f-bb4a-a490e0724623-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:24:57.700699 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.700659    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:57.885718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:57.885688    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:58.053968 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:58.053861    2554 generic.go:356] "Generic (PLEG): container finished" podID="33f4b856-1431-466f-bb4a-a490e0724623" containerID="12aef2e41d0efe414f315619a6e3a5e236d93435b6b2493b02da32f6b6cbdf3d" exitCode=0
Sep 11 13:24:58.053968 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:58.053914    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"
Sep 11 13:24:58.053968 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:58.053950    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s" event={"ID":"33f4b856-1431-466f-bb4a-a490e0724623","Type":"ContainerDied","Data":"12aef2e41d0efe414f315619a6e3a5e236d93435b6b2493b02da32f6b6cbdf3d"}
Sep 11 13:24:58.054538 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:58.054018    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s" event={"ID":"33f4b856-1431-466f-bb4a-a490e0724623","Type":"ContainerDied","Data":"e4581da279ddcaa3877e9bfe63ff4d759860c946b67ca5e5b5eb75d5b01277fb"}
Sep 11 13:24:58.054538 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:58.054046    2554 scope.go:117] "RemoveContainer" containerID="12aef2e41d0efe414f315619a6e3a5e236d93435b6b2493b02da32f6b6cbdf3d"
Sep 11 13:24:58.063456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:58.063433    2554 scope.go:117] "RemoveContainer" containerID="12aef2e41d0efe414f315619a6e3a5e236d93435b6b2493b02da32f6b6cbdf3d"
Sep 11 13:24:58.063765 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:24:58.063742    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12aef2e41d0efe414f315619a6e3a5e236d93435b6b2493b02da32f6b6cbdf3d\": container with ID starting with 12aef2e41d0efe414f315619a6e3a5e236d93435b6b2493b02da32f6b6cbdf3d not found: ID does not exist" containerID="12aef2e41d0efe414f315619a6e3a5e236d93435b6b2493b02da32f6b6cbdf3d"
Sep 11 13:24:58.063832 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:58.063773    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12aef2e41d0efe414f315619a6e3a5e236d93435b6b2493b02da32f6b6cbdf3d"} err="failed to get container status \"12aef2e41d0efe414f315619a6e3a5e236d93435b6b2493b02da32f6b6cbdf3d\": rpc error: code = NotFound desc = could not find container \"12aef2e41d0efe414f315619a6e3a5e236d93435b6b2493b02da32f6b6cbdf3d\": container with ID starting with 12aef2e41d0efe414f315619a6e3a5e236d93435b6b2493b02da32f6b6cbdf3d not found: ID does not exist"
Sep 11 13:24:58.110545 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:58.110504    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"]
Sep 11 13:24:58.130091 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:58.127534    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f78ffd60-79f6d8d559-6427s"]
Sep 11 13:24:58.275617 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:58.275570    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:58.510220 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:58.510170    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:58.812167 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:58.812037    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:59.153010 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:59.152974    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33f4b856-1431-466f-bb4a-a490e0724623" path="/var/lib/kubelet/pods/33f4b856-1431-466f-bb4a-a490e0724623/volumes"
Sep 11 13:24:59.965980 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:24:59.965945    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:02.893130 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:02.893095    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:03.356015 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:03.355979    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:03.611438 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:03.611336    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:03.880374 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:03.880275    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:04.126993 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:04.126959    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:04.515634 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:04.515601    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:05.586634 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:05.586584    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:07.683552 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:07.683515    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:08.203780 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:08.203743    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:08.562429 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:08.562295    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:10.337727 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:10.337665    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:11.091753 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:11.091713    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:11.334049 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:11.334013    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:11.727571 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:11.727534    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:12.072259 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:12.072229    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:12.568289 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:12.568257    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:12.926521 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:12.926438    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:13.258222 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.258130    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:13.266166 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.266138    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"]
Sep 11 13:25:13.266505 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.266491    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33f4b856-1431-466f-bb4a-a490e0724623" containerName="maas-api"
Sep 11 13:25:13.266566 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.266506    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="33f4b856-1431-466f-bb4a-a490e0724623" containerName="maas-api"
Sep 11 13:25:13.266602 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.266577    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="33f4b856-1431-466f-bb4a-a490e0724623" containerName="maas-api"
Sep 11 13:25:13.269661 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.269640    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.272401 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.272356    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-12f386-serving-cert\""
Sep 11 13:25:13.370694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.370659    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"]
Sep 11 13:25:13.442399 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.442334    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.442605 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.442449    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6jlz\" (UniqueName: \"kubernetes.io/projected/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-kube-api-access-w6jlz\") pod \"maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.442605 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.442534    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.442605 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.442577    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.543113 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.543019    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.543113 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.543064    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.543113 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.543099    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.543346 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.543122    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6jlz\" (UniqueName: \"kubernetes.io/projected/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-kube-api-access-w6jlz\") pod \"maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.543914 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.543885    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.545712 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.545692    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.545821 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.545748    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.565176 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.565143    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6jlz\" (UniqueName: \"kubernetes.io/projected/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-kube-api-access-w6jlz\") pod \"maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.580097 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.580069    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:13.615356 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.615321    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:13.800402 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.800286    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:13.878741 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.878708    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"]
Sep 11 13:25:13.879093 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:25:13.879070    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd9a8898_bbe1_4147_bbb9_25c94b80c6c9.slice/crio-483ed9990049d6cd95a1972cb06147c2aeee1f673b93d511150e2d16626d1ca8 WatchSource:0}: Error finding container 483ed9990049d6cd95a1972cb06147c2aeee1f673b93d511150e2d16626d1ca8: Status 404 returned error can't find the container with id 483ed9990049d6cd95a1972cb06147c2aeee1f673b93d511150e2d16626d1ca8
Sep 11 13:25:13.880561 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:13.880540    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:25:14.136535 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:14.136494    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4" event={"ID":"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9","Type":"ContainerStarted","Data":"483ed9990049d6cd95a1972cb06147c2aeee1f673b93d511150e2d16626d1ca8"}
Sep 11 13:25:14.697542 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:14.697456    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:15.142965 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:15.142926    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4" event={"ID":"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9","Type":"ContainerStarted","Data":"a54ed36515a6036adc8c8c96b8023a1107ac24857c3d50943dfd3d23112fda66"}
Sep 11 13:25:15.143426 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:15.143067    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:15.208883 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:15.208830    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4" podStartSLOduration=1.700859333 podStartE2EDuration="2.208814515s" totalImagesPullingTime="507.955182ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:25:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:25:15.206192145 +0000 UTC m=+1238.523649186" watchObservedRunningTime="2026-09-11 13:25:15.208814515 +0000 UTC m=+1238.526271558"
Sep 11 13:25:15.366383 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:15.366332    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:21.156792 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:21.156751    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:25:21.339779 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:21.339746    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:21.699946 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:21.699907    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:22.084536 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:22.084503    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:22.511196 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:22.511113    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:22.934965 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:22.934927    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:30.113606 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:30.113521    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:30.234526 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:30.234490    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:31.045569 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:31.045507    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:31.658578 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:31.658542    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:32.129312 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:32.129276    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:35.410006 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:35.409962    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:36.118250 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:36.118212    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:36.226843 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:36.226801    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:36.454656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:36.454571    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:37.873200 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:37.873161    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:38.869615 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:38.869583    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"]
Sep 11 13:25:38.874099 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:38.874076    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:38.878462 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:38.878427    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-e0ebd7-serving-cert\""
Sep 11 13:25:38.906651 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:38.906622    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"]
Sep 11 13:25:38.934967 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:38.934932    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:38.966536 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:38.966503    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prc99\" (UniqueName: \"kubernetes.io/projected/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-kube-api-access-prc99\") pod \"maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:38.966697 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:38.966557    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:38.966697 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:38.966577    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:38.966697 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:38.966627    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:39.043348 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.043318    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:39.067519 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.067484    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:39.067678 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.067568    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prc99\" (UniqueName: \"kubernetes.io/projected/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-kube-api-access-prc99\") pod \"maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:39.067678 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.067626    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:39.067678 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.067651    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:39.067858 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:25:39.067766    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-serving-cert: secret "maas-api-e2e-shared-a-w2-e0ebd7-serving-cert" not found
Sep 11 13:25:39.067858 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:25:39.067836    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-maas-api-tls podName:c1cf9568-e5f7-462c-b6f6-e35f343b8b2c nodeName:}" failed. No retries permitted until 2026-09-11 13:25:39.567816134 +0000 UTC m=+1262.885273154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-maas-api-tls") pod "maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd" (UID: "c1cf9568-e5f7-462c-b6f6-e35f343b8b2c") : secret "maas-api-e2e-shared-a-w2-e0ebd7-serving-cert" not found
Sep 11 13:25:39.068232 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.068211    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:39.069892 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.069869    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:39.079082 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.079060    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prc99\" (UniqueName: \"kubernetes.io/projected/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-kube-api-access-prc99\") pod \"maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:39.451201 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.451169    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:39.556153 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.556112    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:39.570958 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.570924    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:39.573208 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.573188    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:39.671872 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.671834    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:39.792412 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:39.792320    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:40.079102 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:40.079065    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"]
Sep 11 13:25:40.082151 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:25:40.082123    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1cf9568_e5f7_462c_b6f6_e35f343b8b2c.slice/crio-60556bfd2f506cf7c6dab9ac9a3608aced04a748c2ca74047e8327b5f9f8f76e WatchSource:0}: Error finding container 60556bfd2f506cf7c6dab9ac9a3608aced04a748c2ca74047e8327b5f9f8f76e: Status 404 returned error can't find the container with id 60556bfd2f506cf7c6dab9ac9a3608aced04a748c2ca74047e8327b5f9f8f76e
Sep 11 13:25:40.181492 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:40.181450    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:40.267700 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:40.267660    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd" event={"ID":"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c","Type":"ContainerStarted","Data":"60556bfd2f506cf7c6dab9ac9a3608aced04a748c2ca74047e8327b5f9f8f76e"}
Sep 11 13:25:41.249315 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:41.249276    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:41.274303 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:41.274272    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd" event={"ID":"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c","Type":"ContainerStarted","Data":"f58ddd8e519c38d234b67d7b5d9fd3018d76b93d3061a0d192f0ad38cdbdf911"}
Sep 11 13:25:41.274499 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:41.274329    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:25:41.295805 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:41.295754    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd" podStartSLOduration=2.962615924 podStartE2EDuration="3.295735973s" totalImagesPullingTime="333.120049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:25:41.293934143 +0000 UTC m=+1264.611391184" watchObservedRunningTime="2026-09-11 13:25:41.295735973 +0000 UTC m=+1264.613193013"
Sep 11 13:25:42.023660 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:42.023626    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:42.575476 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:42.575431    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:42.809603 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:42.809574    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:47.285557 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:25:47.285528    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:26:01.039349 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:01.039317    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:01.354191 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:01.354158    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:01.454338 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:01.454307    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:01.710590 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:01.710497    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:02.233020 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:02.232982    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:02.344416 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:02.344307    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:03.250420 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:03.250388    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:04.558342 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:04.558306    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:04.673377 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:04.673331    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:05.460033 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:05.459988    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:06.554390 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:06.554330    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:07.454728 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:07.454677    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:08.491446 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:08.491414    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:08.715415 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:08.715355    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:09.065895 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:09.065857    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:09.153960 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:09.153928    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:11.883209 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:11.883177    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"]
Sep 11 13:26:11.886634 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:11.886617    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:11.889532 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:11.889508    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-6ffb6a\"/\"default-dockercfg-bxnq5\""
Sep 11 13:26:11.889675 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:11.889512    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-6ffb6a\"/\"openshift-service-ca.crt\""
Sep 11 13:26:11.889675 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:11.889572    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-6ffb6a\"/\"kube-root-ca.crt\""
Sep 11 13:26:11.889675 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:11.889641    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-6ffb6a\"/\"auth-test-model-6ffb6a-kserve-self-signed-certs\""
Sep 11 13:26:11.895949 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:11.895926    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"]
Sep 11 13:26:11.998481 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:11.998444    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:12.042754 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.042718    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-home\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.042918 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.042769    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-model-cache\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.042918 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.042819    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k2ln\" (UniqueName: \"kubernetes.io/projected/9cac555e-26c4-4467-93cc-278aa22bc73c-kube-api-access-6k2ln\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.042918 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.042839    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-tmp-dir\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.043031 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.042931    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-dshm\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.043031 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.042968    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cac555e-26c4-4467-93cc-278aa22bc73c-tls-certs\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.144080 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.143988    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cac555e-26c4-4467-93cc-278aa22bc73c-tls-certs\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.144080 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.144040    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-home\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.144080 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.144068    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-model-cache\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.144352 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.144098    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k2ln\" (UniqueName: \"kubernetes.io/projected/9cac555e-26c4-4467-93cc-278aa22bc73c-kube-api-access-6k2ln\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.144352 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.144117    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-tmp-dir\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.144352 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.144168    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-dshm\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.144578 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.144552    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-model-cache\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.144639 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.144585    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-home\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.144639 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.144594    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-tmp-dir\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.146385 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.146349    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-dshm\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.146614 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.146599    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cac555e-26c4-4467-93cc-278aa22bc73c-tls-certs\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.152816 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.152795    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k2ln\" (UniqueName: \"kubernetes.io/projected/9cac555e-26c4-4467-93cc-278aa22bc73c-kube-api-access-6k2ln\") pod \"auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.196924 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.196883    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:12.469864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:12.469839    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"]
Sep 11 13:26:12.471935 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:26:12.471905    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cac555e_26c4_4467_93cc_278aa22bc73c.slice/crio-8132bb6877be9254e731add40aca5db3bd28cab790e80bfece8c204dcc93f70b WatchSource:0}: Error finding container 8132bb6877be9254e731add40aca5db3bd28cab790e80bfece8c204dcc93f70b: Status 404 returned error can't find the container with id 8132bb6877be9254e731add40aca5db3bd28cab790e80bfece8c204dcc93f70b
Sep 11 13:26:13.435336 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:13.435297    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw" event={"ID":"9cac555e-26c4-4467-93cc-278aa22bc73c","Type":"ContainerStarted","Data":"d2e7b45e0f4041c23a38179827a3b31a18560625d8e5bdbe3322c1db0c09929d"}
Sep 11 13:26:13.435336 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:13.435341    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw" event={"ID":"9cac555e-26c4-4467-93cc-278aa22bc73c","Type":"ContainerStarted","Data":"8132bb6877be9254e731add40aca5db3bd28cab790e80bfece8c204dcc93f70b"}
Sep 11 13:26:13.455425 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:13.455352    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw" podStartSLOduration=2.455336077 podStartE2EDuration="2.455336077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:13.45419782 +0000 UTC m=+1296.771654861" watchObservedRunningTime="2026-09-11 13:26:13.455336077 +0000 UTC m=+1296.772793117"
Sep 11 13:26:19.932999 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:19.932963    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:22.197807 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:22.197760    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:22.198304 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:22.197829    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:22.210159 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:22.210131    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:22.493158 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:22.493076    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:26:29.594798 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:29.594765    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:30.932812 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:30.932758    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:31.657271 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:31.657233    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:32.560758 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:32.560724    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:33.363947 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:33.363913    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:34.075890 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:34.075852    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:34.642171 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:34.642139    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:43.456669 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:43.456637    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:43.794462 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:43.794357    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:43.991346 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:43.991316    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:47.407374 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:47.407322    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:47.560262 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:47.560225    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:48.873095 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:48.873061    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:50.158288 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:50.158246    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:51.182905 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:51.182871    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:52.244096 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.244061    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"]
Sep 11 13:26:52.247904 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.247884    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:52.253526 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.253495    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3d5dcb7b-serving-cert\""
Sep 11 13:26:52.291181 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.291149    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"]
Sep 11 13:26:52.294661 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.294630    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f997e61e-ce46-4017-b4ff-65d926eff463-openshift-service-ca\") pod \"maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:52.294829 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.294695    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqqdf\" (UniqueName: \"kubernetes.io/projected/f997e61e-ce46-4017-b4ff-65d926eff463-kube-api-access-vqqdf\") pod \"maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:52.294829 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.294787    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-maas-api-tls\") pod \"maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:52.294924 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.294874    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-metrics-certs\") pod \"maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:52.329463 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.329415    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:52.395652 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.395612    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqqdf\" (UniqueName: \"kubernetes.io/projected/f997e61e-ce46-4017-b4ff-65d926eff463-kube-api-access-vqqdf\") pod \"maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:52.395652 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.395661    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-maas-api-tls\") pod \"maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:52.395925 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.395703    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-metrics-certs\") pod \"maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:52.395925 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.395732    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f997e61e-ce46-4017-b4ff-65d926eff463-openshift-service-ca\") pod \"maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:52.395925 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:26:52.395823    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-serving-cert: secret "maas-api-e2e-mt-3d5dcb7b-serving-cert" not found
Sep 11 13:26:52.395925 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:26:52.395894    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-maas-api-tls podName:f997e61e-ce46-4017-b4ff-65d926eff463 nodeName:}" failed. No retries permitted until 2026-09-11 13:26:52.895876618 +0000 UTC m=+1336.213333642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-maas-api-tls") pod "maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs" (UID: "f997e61e-ce46-4017-b4ff-65d926eff463") : secret "maas-api-e2e-mt-3d5dcb7b-serving-cert" not found
Sep 11 13:26:52.396392 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.396346    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f997e61e-ce46-4017-b4ff-65d926eff463-openshift-service-ca\") pod \"maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:52.398252 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.398227    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-metrics-certs\") pod \"maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:52.415053 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.415017    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqqdf\" (UniqueName: \"kubernetes.io/projected/f997e61e-ce46-4017-b4ff-65d926eff463-kube-api-access-vqqdf\") pod \"maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:52.567863 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.567821    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:52.785615 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.785563    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:52.902600 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.902499    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-maas-api-tls\") pod \"maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:52.905069 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:52.905043    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-maas-api-tls\") pod \"maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:53.158865 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:53.158771    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:53.189718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:53.189687    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:53.309142 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:53.309105    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:53.454914 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:53.454889    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"]
Sep 11 13:26:53.457625 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:26:53.457594    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf997e61e_ce46_4017_b4ff_65d926eff463.slice/crio-7649f60e92d996b240246930c067460a9d882d5fd50eeb6c27ab8f0204cc0475 WatchSource:0}: Error finding container 7649f60e92d996b240246930c067460a9d882d5fd50eeb6c27ab8f0204cc0475: Status 404 returned error can't find the container with id 7649f60e92d996b240246930c067460a9d882d5fd50eeb6c27ab8f0204cc0475
Sep 11 13:26:53.646347 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:53.646309    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs" event={"ID":"f997e61e-ce46-4017-b4ff-65d926eff463","Type":"ContainerStarted","Data":"7649f60e92d996b240246930c067460a9d882d5fd50eeb6c27ab8f0204cc0475"}
Sep 11 13:26:54.653233 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:54.653191    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs" event={"ID":"f997e61e-ce46-4017-b4ff-65d926eff463","Type":"ContainerStarted","Data":"65dab9a2fcc924c895c12d4287f82a98449d05d273af0d6d2bfd3dcf3a4ec3f6"}
Sep 11 13:26:54.653691 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:54.653326    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:26:54.689116 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:54.689060    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs" podStartSLOduration=2.2753531909999998 podStartE2EDuration="2.689042918s" totalImagesPullingTime="413.689727ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:54.688119671 +0000 UTC m=+1338.005576732" watchObservedRunningTime="2026-09-11 13:26:54.689042918 +0000 UTC m=+1338.006499958"
Sep 11 13:26:58.290799 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:26:58.290756    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:00.664108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:00.664036    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:27:03.750202 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:03.750163    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:16.751272 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:16.751206    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:17.397165 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:17.397132    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:17.864698 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:17.864662    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:18.623044 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:18.623009    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:19.543619 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:19.543588    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:19.644917 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:19.644882    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:19.743666 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:19.743633    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:20.483482 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:20.483434    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:21.331549 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:21.331503    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:26.083477 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.083438    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"]
Sep 11 13:27:26.087230 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.087210    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.090587 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.090564    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3d5dcb7b\"/\"kube-root-ca.crt\""
Sep 11 13:27:26.090822 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.090799    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3d5dcb7b\"/\"openshift-service-ca.crt\""
Sep 11 13:27:26.091421 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.091403    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3d5dcb7b\"/\"e2e-lifecycle-model-3d5dcb7b-kserve-self-signed-certs\""
Sep 11 13:27:26.091491 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.091429    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3d5dcb7b\"/\"default-dockercfg-lfqdf\""
Sep 11 13:27:26.099174 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.099147    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-home\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.099291 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.099180    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-tmp-dir\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.099291 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.099198    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-model-cache\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.099410 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.099291    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27mrf\" (UniqueName: \"kubernetes.io/projected/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-kube-api-access-27mrf\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.099410 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.099318    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-tls-certs\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.099410 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.099344    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-dshm\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.115942 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.115900    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"]
Sep 11 13:27:26.131610 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.131576    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:26.200412 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.200337    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-model-cache\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.200594 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.200511    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27mrf\" (UniqueName: \"kubernetes.io/projected/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-kube-api-access-27mrf\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.200594 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.200565    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-tls-certs\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.200733 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.200621    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-dshm\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.200846 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.200817    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-model-cache\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.201026 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.200826    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-home\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.201108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.201079    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-tmp-dir\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.201170 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.201105    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-home\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.201403 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.201359    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-tmp-dir\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.203541 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.203510    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-dshm\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.203755 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.203740    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-tls-certs\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.210625 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.210598    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27mrf\" (UniqueName: \"kubernetes.io/projected/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-kube-api-access-27mrf\") pod \"e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") " pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.397171 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.397077    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:26.679826 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.679747    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"]
Sep 11 13:27:26.682920 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:27:26.682889    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e53b6a5_8ccd_4d90_9aa0_6f91ad1f2492.slice/crio-23b2c013485d8a2d9e4a3d5a80306e0b058902be6027828106f6cd2202d8eab4 WatchSource:0}: Error finding container 23b2c013485d8a2d9e4a3d5a80306e0b058902be6027828106f6cd2202d8eab4: Status 404 returned error can't find the container with id 23b2c013485d8a2d9e4a3d5a80306e0b058902be6027828106f6cd2202d8eab4
Sep 11 13:27:26.817266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.817227    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb" event={"ID":"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492","Type":"ContainerStarted","Data":"02165dc4697d6f106fbcc09e9f394e7ba48da781c685de6dc3636eb629ea75b0"}
Sep 11 13:27:26.817480 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.817274    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb" event={"ID":"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492","Type":"ContainerStarted","Data":"23b2c013485d8a2d9e4a3d5a80306e0b058902be6027828106f6cd2202d8eab4"}
Sep 11 13:27:26.847806 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:26.847746    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb" podStartSLOduration=0.847727457 podStartE2EDuration="847.727457ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:27:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:27:26.844154739 +0000 UTC m=+1370.161611779" watchObservedRunningTime="2026-09-11 13:27:26.847727457 +0000 UTC m=+1370.165184497"
Sep 11 13:27:36.397897 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:36.397850    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:36.398535 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:36.397926    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:36.410959 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:36.410926    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:27:36.880374 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:27:36.880325    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:28:02.531698 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:02.531664    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:03.377977 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:03.377934    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:04.186094 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:04.186055    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:04.926740 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:04.926708    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:05.963977 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:05.963938    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:06.999096 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:06.999066    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:17.359541 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.359503    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"]
Sep 11 13:28:17.360088 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.359753    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs" podUID="f997e61e-ce46-4017-b4ff-65d926eff463" containerName="maas-api" containerID="cri-o://65dab9a2fcc924c895c12d4287f82a98449d05d273af0d6d2bfd3dcf3a4ec3f6" gracePeriod=30
Sep 11 13:28:17.419652 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.419615    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:17.532122 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.532086    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:17.611896 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.611830    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:28:17.628520 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.628485    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:17.659694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.659656    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-maas-api-tls\") pod \"f997e61e-ce46-4017-b4ff-65d926eff463\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") "
Sep 11 13:28:17.659883 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.659709    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-metrics-certs\") pod \"f997e61e-ce46-4017-b4ff-65d926eff463\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") "
Sep 11 13:28:17.659883 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.659785    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f997e61e-ce46-4017-b4ff-65d926eff463-openshift-service-ca\") pod \"f997e61e-ce46-4017-b4ff-65d926eff463\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") "
Sep 11 13:28:17.659883 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.659829    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqqdf\" (UniqueName: \"kubernetes.io/projected/f997e61e-ce46-4017-b4ff-65d926eff463-kube-api-access-vqqdf\") pod \"f997e61e-ce46-4017-b4ff-65d926eff463\" (UID: \"f997e61e-ce46-4017-b4ff-65d926eff463\") "
Sep 11 13:28:17.660236 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.660207    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f997e61e-ce46-4017-b4ff-65d926eff463-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f997e61e-ce46-4017-b4ff-65d926eff463" (UID: "f997e61e-ce46-4017-b4ff-65d926eff463"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:28:17.662170 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.662139    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f997e61e-ce46-4017-b4ff-65d926eff463" (UID: "f997e61e-ce46-4017-b4ff-65d926eff463"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:28:17.662279 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.662192    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f997e61e-ce46-4017-b4ff-65d926eff463-kube-api-access-vqqdf" (OuterVolumeSpecName: "kube-api-access-vqqdf") pod "f997e61e-ce46-4017-b4ff-65d926eff463" (UID: "f997e61e-ce46-4017-b4ff-65d926eff463"). InnerVolumeSpecName "kube-api-access-vqqdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:28:17.662279 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.662221    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f997e61e-ce46-4017-b4ff-65d926eff463" (UID: "f997e61e-ce46-4017-b4ff-65d926eff463"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:28:17.706200 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.706146    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:17.760684 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.760642    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f997e61e-ce46-4017-b4ff-65d926eff463-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:28:17.760684 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.760687    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqqdf\" (UniqueName: \"kubernetes.io/projected/f997e61e-ce46-4017-b4ff-65d926eff463-kube-api-access-vqqdf\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:28:17.760890 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.760702    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:28:17.760890 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:17.760716    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f997e61e-ce46-4017-b4ff-65d926eff463-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:28:18.096455 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:18.096415    2554 generic.go:356] "Generic (PLEG): container finished" podID="f997e61e-ce46-4017-b4ff-65d926eff463" containerID="65dab9a2fcc924c895c12d4287f82a98449d05d273af0d6d2bfd3dcf3a4ec3f6" exitCode=0
Sep 11 13:28:18.096636 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:18.096469    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"
Sep 11 13:28:18.096636 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:18.096480    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs" event={"ID":"f997e61e-ce46-4017-b4ff-65d926eff463","Type":"ContainerDied","Data":"65dab9a2fcc924c895c12d4287f82a98449d05d273af0d6d2bfd3dcf3a4ec3f6"}
Sep 11 13:28:18.096636 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:18.096522    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs" event={"ID":"f997e61e-ce46-4017-b4ff-65d926eff463","Type":"ContainerDied","Data":"7649f60e92d996b240246930c067460a9d882d5fd50eeb6c27ab8f0204cc0475"}
Sep 11 13:28:18.096636 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:18.096545    2554 scope.go:117] "RemoveContainer" containerID="65dab9a2fcc924c895c12d4287f82a98449d05d273af0d6d2bfd3dcf3a4ec3f6"
Sep 11 13:28:18.106082 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:18.106062    2554 scope.go:117] "RemoveContainer" containerID="65dab9a2fcc924c895c12d4287f82a98449d05d273af0d6d2bfd3dcf3a4ec3f6"
Sep 11 13:28:18.106410 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:28:18.106389    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65dab9a2fcc924c895c12d4287f82a98449d05d273af0d6d2bfd3dcf3a4ec3f6\": container with ID starting with 65dab9a2fcc924c895c12d4287f82a98449d05d273af0d6d2bfd3dcf3a4ec3f6 not found: ID does not exist" containerID="65dab9a2fcc924c895c12d4287f82a98449d05d273af0d6d2bfd3dcf3a4ec3f6"
Sep 11 13:28:18.106461 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:18.106420    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65dab9a2fcc924c895c12d4287f82a98449d05d273af0d6d2bfd3dcf3a4ec3f6"} err="failed to get container status \"65dab9a2fcc924c895c12d4287f82a98449d05d273af0d6d2bfd3dcf3a4ec3f6\": rpc error: code = NotFound desc = could not find container \"65dab9a2fcc924c895c12d4287f82a98449d05d273af0d6d2bfd3dcf3a4ec3f6\": container with ID starting with 65dab9a2fcc924c895c12d4287f82a98449d05d273af0d6d2bfd3dcf3a4ec3f6 not found: ID does not exist"
Sep 11 13:28:18.112976 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:18.112956    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:18.121643 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:18.121618    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"]
Sep 11 13:28:18.124844 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:18.124817    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3d5dcb7b-6f9794d66b-d5mrs"]
Sep 11 13:28:18.848312 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:18.848278    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:19.153554 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:19.153476    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f997e61e-ce46-4017-b4ff-65d926eff463" path="/var/lib/kubelet/pods/f997e61e-ce46-4017-b4ff-65d926eff463/volumes"
Sep 11 13:28:19.836925 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:19.836891    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:20.575208 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:20.575165    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:20.695609 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:20.695576    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:21.317699 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:21.317668    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:25.672474 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:25.672437    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"]
Sep 11 13:28:25.728191 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:25.728155    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:32.355047 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:32.355005    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:32.423130 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:32.423097    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:40.692092 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:40.692014    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb" podUID="4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492" containerName="main" containerID="cri-o://02165dc4697d6f106fbcc09e9f394e7ba48da781c685de6dc3636eb629ea75b0" gracePeriod=45
Sep 11 13:28:40.944970 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:40.944899    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:28:41.066282 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.066250    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27mrf\" (UniqueName: \"kubernetes.io/projected/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-kube-api-access-27mrf\") pod \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") "
Sep 11 13:28:41.066282 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.066288    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-tmp-dir\") pod \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") "
Sep 11 13:28:41.066514 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.066306    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-home\") pod \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") "
Sep 11 13:28:41.066514 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.066405    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-model-cache\") pod \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") "
Sep 11 13:28:41.066514 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.066444    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-tls-certs\") pod \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") "
Sep 11 13:28:41.066514 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.066471    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-dshm\") pod \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\" (UID: \"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492\") "
Sep 11 13:28:41.066728 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.066569    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492" (UID: "4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:28:41.066728 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.066621    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-home" (OuterVolumeSpecName: "home") pod "4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492" (UID: "4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:28:41.066728 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.066637    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492" (UID: "4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:28:41.066876 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.066773    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:28:41.066876 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.066787    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:28:41.066876 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.066797    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:28:41.068567 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.068535    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492" (UID: "4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:28:41.068567 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.068537    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-dshm" (OuterVolumeSpecName: "dshm") pod "4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492" (UID: "4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:28:41.068718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.068598    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-kube-api-access-27mrf" (OuterVolumeSpecName: "kube-api-access-27mrf") pod "4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492" (UID: "4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492"). InnerVolumeSpecName "kube-api-access-27mrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:28:41.167616 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.167591    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:28:41.167616 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.167615    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:28:41.167760 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.167624    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27mrf\" (UniqueName: \"kubernetes.io/projected/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492-kube-api-access-27mrf\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:28:41.215252 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.215158    2554 generic.go:356] "Generic (PLEG): container finished" podID="4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492" containerID="02165dc4697d6f106fbcc09e9f394e7ba48da781c685de6dc3636eb629ea75b0" exitCode=0
Sep 11 13:28:41.215252 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.215208    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"
Sep 11 13:28:41.215252 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.215240    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb" event={"ID":"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492","Type":"ContainerDied","Data":"02165dc4697d6f106fbcc09e9f394e7ba48da781c685de6dc3636eb629ea75b0"}
Sep 11 13:28:41.215555 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.215286    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb" event={"ID":"4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492","Type":"ContainerDied","Data":"23b2c013485d8a2d9e4a3d5a80306e0b058902be6027828106f6cd2202d8eab4"}
Sep 11 13:28:41.215555 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.215304    2554 scope.go:117] "RemoveContainer" containerID="02165dc4697d6f106fbcc09e9f394e7ba48da781c685de6dc3636eb629ea75b0"
Sep 11 13:28:41.223854 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.223838    2554 scope.go:117] "RemoveContainer" containerID="02165dc4697d6f106fbcc09e9f394e7ba48da781c685de6dc3636eb629ea75b0"
Sep 11 13:28:41.224134 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:28:41.224111    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02165dc4697d6f106fbcc09e9f394e7ba48da781c685de6dc3636eb629ea75b0\": container with ID starting with 02165dc4697d6f106fbcc09e9f394e7ba48da781c685de6dc3636eb629ea75b0 not found: ID does not exist" containerID="02165dc4697d6f106fbcc09e9f394e7ba48da781c685de6dc3636eb629ea75b0"
Sep 11 13:28:41.224183 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.224142    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02165dc4697d6f106fbcc09e9f394e7ba48da781c685de6dc3636eb629ea75b0"} err="failed to get container status \"02165dc4697d6f106fbcc09e9f394e7ba48da781c685de6dc3636eb629ea75b0\": rpc error: code = NotFound desc = could not find container \"02165dc4697d6f106fbcc09e9f394e7ba48da781c685de6dc3636eb629ea75b0\": container with ID starting with 02165dc4697d6f106fbcc09e9f394e7ba48da781c685de6dc3636eb629ea75b0 not found: ID does not exist"
Sep 11 13:28:41.232726 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.232703    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"]
Sep 11 13:28:41.236875 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:41.236854    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-3d5dcb7b/e2e-lifecycle-model-3d5dcb7b-kserve-69f5778d75-nr7bb"]
Sep 11 13:28:43.153266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:28:43.153221    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492" path="/var/lib/kubelet/pods/4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492/volumes"
Sep 11 13:29:02.221500 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:02.221465    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:03.279989 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:03.279953    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:09.421612 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:09.421575    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"]
Sep 11 13:29:09.447021 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:09.446985    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:10.266506 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:10.266464    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:10.377238 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:10.377202    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:10.545226 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:10.545139    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:10.663433 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:10.663403    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:11.867740 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:11.867701    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:11.967228 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:11.967185    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:12.828577 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:12.828528    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:13.003535 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:13.003500    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:14.432983 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:14.432949    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:20.866128 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:20.866097    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:20.971295 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:20.971256    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:21.536379 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:21.536345    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:24.440173 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.440131    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw" podUID="9cac555e-26c4-4467-93cc-278aa22bc73c" containerName="main" containerID="cri-o://d2e7b45e0f4041c23a38179827a3b31a18560625d8e5bdbe3322c1db0c09929d" gracePeriod=45
Sep 11 13:29:24.685316 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.685291    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:29:24.759866 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.759777    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cac555e-26c4-4467-93cc-278aa22bc73c-tls-certs\") pod \"9cac555e-26c4-4467-93cc-278aa22bc73c\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") "
Sep 11 13:29:24.759866 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.759854    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-tmp-dir\") pod \"9cac555e-26c4-4467-93cc-278aa22bc73c\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") "
Sep 11 13:29:24.760055 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.759880    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-home\") pod \"9cac555e-26c4-4467-93cc-278aa22bc73c\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") "
Sep 11 13:29:24.760055 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.759983    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k2ln\" (UniqueName: \"kubernetes.io/projected/9cac555e-26c4-4467-93cc-278aa22bc73c-kube-api-access-6k2ln\") pod \"9cac555e-26c4-4467-93cc-278aa22bc73c\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") "
Sep 11 13:29:24.760055 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.760011    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-model-cache\") pod \"9cac555e-26c4-4467-93cc-278aa22bc73c\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") "
Sep 11 13:29:24.760055 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.760033    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-dshm\") pod \"9cac555e-26c4-4467-93cc-278aa22bc73c\" (UID: \"9cac555e-26c4-4467-93cc-278aa22bc73c\") "
Sep 11 13:29:24.760258 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.760159    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9cac555e-26c4-4467-93cc-278aa22bc73c" (UID: "9cac555e-26c4-4467-93cc-278aa22bc73c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:24.760258 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.760216    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-home" (OuterVolumeSpecName: "home") pod "9cac555e-26c4-4467-93cc-278aa22bc73c" (UID: "9cac555e-26c4-4467-93cc-278aa22bc73c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:24.760397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.760313    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-model-cache" (OuterVolumeSpecName: "model-cache") pod "9cac555e-26c4-4467-93cc-278aa22bc73c" (UID: "9cac555e-26c4-4467-93cc-278aa22bc73c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:24.760397 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.760352    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:29:24.760480 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.760397    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:29:24.762165 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.762132    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cac555e-26c4-4467-93cc-278aa22bc73c-kube-api-access-6k2ln" (OuterVolumeSpecName: "kube-api-access-6k2ln") pod "9cac555e-26c4-4467-93cc-278aa22bc73c" (UID: "9cac555e-26c4-4467-93cc-278aa22bc73c"). InnerVolumeSpecName "kube-api-access-6k2ln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:29:24.762165 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.762157    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cac555e-26c4-4467-93cc-278aa22bc73c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9cac555e-26c4-4467-93cc-278aa22bc73c" (UID: "9cac555e-26c4-4467-93cc-278aa22bc73c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:29:24.762430 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.762182    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-dshm" (OuterVolumeSpecName: "dshm") pod "9cac555e-26c4-4467-93cc-278aa22bc73c" (UID: "9cac555e-26c4-4467-93cc-278aa22bc73c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:24.861581 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.861542    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k2ln\" (UniqueName: \"kubernetes.io/projected/9cac555e-26c4-4467-93cc-278aa22bc73c-kube-api-access-6k2ln\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:29:24.861581 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.861575    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:29:24.861581 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.861585    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cac555e-26c4-4467-93cc-278aa22bc73c-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:29:24.861581 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:24.861593    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cac555e-26c4-4467-93cc-278aa22bc73c-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:29:25.442166 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:25.442080    2554 generic.go:356] "Generic (PLEG): container finished" podID="9cac555e-26c4-4467-93cc-278aa22bc73c" containerID="d2e7b45e0f4041c23a38179827a3b31a18560625d8e5bdbe3322c1db0c09929d" exitCode=0
Sep 11 13:29:25.442166 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:25.442135    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"
Sep 11 13:29:25.442695 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:25.442168    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw" event={"ID":"9cac555e-26c4-4467-93cc-278aa22bc73c","Type":"ContainerDied","Data":"d2e7b45e0f4041c23a38179827a3b31a18560625d8e5bdbe3322c1db0c09929d"}
Sep 11 13:29:25.442695 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:25.442220    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw" event={"ID":"9cac555e-26c4-4467-93cc-278aa22bc73c","Type":"ContainerDied","Data":"8132bb6877be9254e731add40aca5db3bd28cab790e80bfece8c204dcc93f70b"}
Sep 11 13:29:25.442695 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:25.442244    2554 scope.go:117] "RemoveContainer" containerID="d2e7b45e0f4041c23a38179827a3b31a18560625d8e5bdbe3322c1db0c09929d"
Sep 11 13:29:25.451108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:25.451092    2554 scope.go:117] "RemoveContainer" containerID="d2e7b45e0f4041c23a38179827a3b31a18560625d8e5bdbe3322c1db0c09929d"
Sep 11 13:29:25.451395 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:29:25.451357    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2e7b45e0f4041c23a38179827a3b31a18560625d8e5bdbe3322c1db0c09929d\": container with ID starting with d2e7b45e0f4041c23a38179827a3b31a18560625d8e5bdbe3322c1db0c09929d not found: ID does not exist" containerID="d2e7b45e0f4041c23a38179827a3b31a18560625d8e5bdbe3322c1db0c09929d"
Sep 11 13:29:25.451489 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:25.451402    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2e7b45e0f4041c23a38179827a3b31a18560625d8e5bdbe3322c1db0c09929d"} err="failed to get container status \"d2e7b45e0f4041c23a38179827a3b31a18560625d8e5bdbe3322c1db0c09929d\": rpc error: code = NotFound desc = could not find container \"d2e7b45e0f4041c23a38179827a3b31a18560625d8e5bdbe3322c1db0c09929d\": container with ID starting with d2e7b45e0f4041c23a38179827a3b31a18560625d8e5bdbe3322c1db0c09929d not found: ID does not exist"
Sep 11 13:29:25.460780 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:25.460758    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"]
Sep 11 13:29:25.463408 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:25.463388    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/auth-test-model-6ffb6a-kserve-d5fc89f99-clzdw"]
Sep 11 13:29:27.154144 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:27.154111    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cac555e-26c4-4467-93cc-278aa22bc73c" path="/var/lib/kubelet/pods/9cac555e-26c4-4467-93cc-278aa22bc73c/volumes"
Sep 11 13:29:29.358731 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:29.358700    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:29.466790 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:29.466753    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:35.251538 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:35.251499    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:35.374310 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:35.374275    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:37.190926 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:37.190889    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:29:37.192615 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:37.192591    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:29:39.114421 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:39.114383    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:39.367718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:39.367627    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:41.512165 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:41.512124    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:59.105869 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:29:59.105829    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:00.166377 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.166340    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6ffb6a-29818890-4mh2j"]
Sep 11 13:30:00.166747 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.166694    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f997e61e-ce46-4017-b4ff-65d926eff463" containerName="maas-api"
Sep 11 13:30:00.166747 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.166705    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f997e61e-ce46-4017-b4ff-65d926eff463" containerName="maas-api"
Sep 11 13:30:00.166747 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.166715    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cac555e-26c4-4467-93cc-278aa22bc73c" containerName="main"
Sep 11 13:30:00.166747 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.166720    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cac555e-26c4-4467-93cc-278aa22bc73c" containerName="main"
Sep 11 13:30:00.166747 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.166732    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492" containerName="main"
Sep 11 13:30:00.166747 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.166737    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492" containerName="main"
Sep 11 13:30:00.166948 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.166799    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f997e61e-ce46-4017-b4ff-65d926eff463" containerName="maas-api"
Sep 11 13:30:00.166948 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.166807    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cac555e-26c4-4467-93cc-278aa22bc73c" containerName="main"
Sep 11 13:30:00.166948 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.166814    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e53b6a5-8ccd-4d90-9aa0-6f91ad1f2492" containerName="main"
Sep 11 13:30:00.169592 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.169577    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6ffb6a-29818890-4mh2j"
Sep 11 13:30:00.172439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.172415    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:30:00.178904 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.178884    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6ffb6a-29818890-4mh2j"
Sep 11 13:30:00.195188 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.195160    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6ffb6a-29818890-4mh2j"]
Sep 11 13:30:00.433265 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.433166    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:00.456419 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.456357    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6ffb6a-29818890-4mh2j"]
Sep 11 13:30:00.459599 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:30:00.459573    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde60e156_29f1_41cb_b76d_16d5c733875b.slice/crio-16d4e8e3f9c650b1fa4dafcdd0111edca01d839588be6b0dbf8f2c88347b754c WatchSource:0}: Error finding container 16d4e8e3f9c650b1fa4dafcdd0111edca01d839588be6b0dbf8f2c88347b754c: Status 404 returned error can't find the container with id 16d4e8e3f9c650b1fa4dafcdd0111edca01d839588be6b0dbf8f2c88347b754c
Sep 11 13:30:00.625299 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:00.625263    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6ffb6a-29818890-4mh2j" event={"ID":"de60e156-29f1-41cb-b76d-16d5c733875b","Type":"ContainerStarted","Data":"16d4e8e3f9c650b1fa4dafcdd0111edca01d839588be6b0dbf8f2c88347b754c"}
Sep 11 13:30:01.631910 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:01.631875    2554 generic.go:356] "Generic (PLEG): container finished" podID="de60e156-29f1-41cb-b76d-16d5c733875b" containerID="28acab3e64a4eb8f77fdcc6dd2baf15e0f9ee8d9bd5910b34c27dc4a3c1314de" exitCode=0
Sep 11 13:30:01.632295 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:01.631965    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6ffb6a-29818890-4mh2j" event={"ID":"de60e156-29f1-41cb-b76d-16d5c733875b","Type":"ContainerDied","Data":"28acab3e64a4eb8f77fdcc6dd2baf15e0f9ee8d9bd5910b34c27dc4a3c1314de"}
Sep 11 13:30:02.767120 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:02.767094    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6ffb6a-29818890-4mh2j"
Sep 11 13:30:03.643437 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:03.643406    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6ffb6a-29818890-4mh2j"
Sep 11 13:30:03.643609 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:03.643399    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6ffb6a-29818890-4mh2j" event={"ID":"de60e156-29f1-41cb-b76d-16d5c733875b","Type":"ContainerDied","Data":"16d4e8e3f9c650b1fa4dafcdd0111edca01d839588be6b0dbf8f2c88347b754c"}
Sep 11 13:30:03.643609 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:03.643530    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16d4e8e3f9c650b1fa4dafcdd0111edca01d839588be6b0dbf8f2c88347b754c"
Sep 11 13:30:07.610651 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:07.610619    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:07.813487 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:07.813458    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:07.967462 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:07.967349    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:08.277304 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:08.277200    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:08.561468 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:08.561384    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:10.325753 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.325719    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:10.900025 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.899981    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"]
Sep 11 13:30:10.900911 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.900881    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de60e156-29f1-41cb-b76d-16d5c733875b" containerName="cleanup"
Sep 11 13:30:10.901077 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.901064    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="de60e156-29f1-41cb-b76d-16d5c733875b" containerName="cleanup"
Sep 11 13:30:10.901432 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.901416    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="de60e156-29f1-41cb-b76d-16d5c733875b" containerName="cleanup"
Sep 11 13:30:10.905634 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.905609    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:10.908150 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.908129    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-12f386\"/\"kube-root-ca.crt\""
Sep 11 13:30:10.909927 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.909896    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-12f386\"/\"default-dockercfg-285dr\""
Sep 11 13:30:10.910223 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.910195    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-12f386\"/\"openshift-service-ca.crt\""
Sep 11 13:30:10.910429 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.910270    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-12f386\"/\"sub-test-model-12f386-kserve-self-signed-certs\""
Sep 11 13:30:10.912548 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.912529    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"]
Sep 11 13:30:10.928681 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.928640    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:10.968732 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.968695    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce62669f-837e-41fb-9387-fc6d60a633f0-tls-certs\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:10.968918 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.968734    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-dshm\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:10.968918 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.968824    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-home\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:10.968918 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.968893    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-model-cache\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:10.969045 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.968951    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-tmp-dir\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:10.969045 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:10.968996    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbrds\" (UniqueName: \"kubernetes.io/projected/ce62669f-837e-41fb-9387-fc6d60a633f0-kube-api-access-qbrds\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.067834 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.067792    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:11.069967 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.069943    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbrds\" (UniqueName: \"kubernetes.io/projected/ce62669f-837e-41fb-9387-fc6d60a633f0-kube-api-access-qbrds\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.070080 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.069981    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce62669f-837e-41fb-9387-fc6d60a633f0-tls-certs\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.070080 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.070010    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-dshm\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.070080 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.070035    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-home\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.070080 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.070076    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-model-cache\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.070288 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.070208    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-tmp-dir\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.070581 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.070550    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-home\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.070702 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.070595    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-model-cache\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.070702 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.070620    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-tmp-dir\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.072347 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.072322    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-dshm\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.072471 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.072453    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce62669f-837e-41fb-9387-fc6d60a633f0-tls-certs\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.093152 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.093119    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbrds\" (UniqueName: \"kubernetes.io/projected/ce62669f-837e-41fb-9387-fc6d60a633f0-kube-api-access-qbrds\") pod \"sub-test-model-12f386-kserve-bb76b6987-79hql\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") " pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.211025 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.210939    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:11.218054 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.218019    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:11.343253 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.343214    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:11.497101 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.496974    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"]
Sep 11 13:30:11.508927 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:30:11.508896    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce62669f_837e_41fb_9387_fc6d60a633f0.slice/crio-68cf1df618c98f67411dfe4c3161c9b72133a42388c9c1b3cccc628785413d8e WatchSource:0}: Error finding container 68cf1df618c98f67411dfe4c3161c9b72133a42388c9c1b3cccc628785413d8e: Status 404 returned error can't find the container with id 68cf1df618c98f67411dfe4c3161c9b72133a42388c9c1b3cccc628785413d8e
Sep 11 13:30:11.685327 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.685283    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql" event={"ID":"ce62669f-837e-41fb-9387-fc6d60a633f0","Type":"ContainerStarted","Data":"164265cc84cb282424d67643594509692d929e1bb595d21833fa45b96f3c78e3"}
Sep 11 13:30:11.685555 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.685340    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql" event={"ID":"ce62669f-837e-41fb-9387-fc6d60a633f0","Type":"ContainerStarted","Data":"68cf1df618c98f67411dfe4c3161c9b72133a42388c9c1b3cccc628785413d8e"}
Sep 11 13:30:11.722500 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:11.722444    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql" podStartSLOduration=1.722423435 podStartE2EDuration="1.722423435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:11.721045883 +0000 UTC m=+1535.038502927" watchObservedRunningTime="2026-09-11 13:30:11.722423435 +0000 UTC m=+1535.039880477"
Sep 11 13:30:12.531259 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:12.531222    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:13.353528 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:13.353482    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:14.113443 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:14.113402    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:15.025956 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.025924    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"]
Sep 11 13:30:15.029308 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.029291    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.032062 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.032042    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-0326b6-serving-cert\""
Sep 11 13:30:15.055126 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.055104    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"]
Sep 11 13:30:15.109084 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.109048    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-maas-api-tls\") pod \"maas-api-e2e-api-a-0326b6-7b5574848d-qsf46\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.109253 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.109131    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-metrics-certs\") pod \"maas-api-e2e-api-a-0326b6-7b5574848d-qsf46\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.109253 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.109164    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlxxz\" (UniqueName: \"kubernetes.io/projected/4c1679f3-5ebe-47de-948e-244f1b3edee9-kube-api-access-rlxxz\") pod \"maas-api-e2e-api-a-0326b6-7b5574848d-qsf46\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.109253 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.109217    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c1679f3-5ebe-47de-948e-244f1b3edee9-openshift-service-ca\") pod \"maas-api-e2e-api-a-0326b6-7b5574848d-qsf46\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.114285 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.114255    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:15.209929 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.209881    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-maas-api-tls\") pod \"maas-api-e2e-api-a-0326b6-7b5574848d-qsf46\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.210120 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.209950    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-metrics-certs\") pod \"maas-api-e2e-api-a-0326b6-7b5574848d-qsf46\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.210120 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.209978    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlxxz\" (UniqueName: \"kubernetes.io/projected/4c1679f3-5ebe-47de-948e-244f1b3edee9-kube-api-access-rlxxz\") pod \"maas-api-e2e-api-a-0326b6-7b5574848d-qsf46\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.210120 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.210015    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c1679f3-5ebe-47de-948e-244f1b3edee9-openshift-service-ca\") pod \"maas-api-e2e-api-a-0326b6-7b5574848d-qsf46\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.210120 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:30:15.210052    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-serving-cert: secret "maas-api-e2e-api-a-0326b6-serving-cert" not found
Sep 11 13:30:15.210286 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:30:15.210132    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-maas-api-tls podName:4c1679f3-5ebe-47de-948e-244f1b3edee9 nodeName:}" failed. No retries permitted until 2026-09-11 13:30:15.710110368 +0000 UTC m=+1539.027567386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-maas-api-tls") pod "maas-api-e2e-api-a-0326b6-7b5574848d-qsf46" (UID: "4c1679f3-5ebe-47de-948e-244f1b3edee9") : secret "maas-api-e2e-api-a-0326b6-serving-cert" not found
Sep 11 13:30:15.210775 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.210752    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c1679f3-5ebe-47de-948e-244f1b3edee9-openshift-service-ca\") pod \"maas-api-e2e-api-a-0326b6-7b5574848d-qsf46\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.212463 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.212434    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-metrics-certs\") pod \"maas-api-e2e-api-a-0326b6-7b5574848d-qsf46\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.217355 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.217330    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:15.220713 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.220687    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlxxz\" (UniqueName: \"kubernetes.io/projected/4c1679f3-5ebe-47de-948e-244f1b3edee9-kube-api-access-rlxxz\") pod \"maas-api-e2e-api-a-0326b6-7b5574848d-qsf46\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.602570 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.602531    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:15.714265 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.714231    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-maas-api-tls\") pod \"maas-api-e2e-api-a-0326b6-7b5574848d-qsf46\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.716733 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.716708    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-maas-api-tls\") pod \"maas-api-e2e-api-a-0326b6-7b5574848d-qsf46\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:15.745732 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.745692    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:15.849894 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.849858    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:15.939945 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:15.939866    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:16.225076 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:16.225044    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"]
Sep 11 13:30:16.227670 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:30:16.227640    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c1679f3_5ebe_47de_948e_244f1b3edee9.slice/crio-94779940e5226e2780fabeb306e7336a8226012a45f404e88d95de1aefe0ed12 WatchSource:0}: Error finding container 94779940e5226e2780fabeb306e7336a8226012a45f404e88d95de1aefe0ed12: Status 404 returned error can't find the container with id 94779940e5226e2780fabeb306e7336a8226012a45f404e88d95de1aefe0ed12
Sep 11 13:30:16.229045 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:16.229025    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:30:16.713954 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:16.713919    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46" event={"ID":"4c1679f3-5ebe-47de-948e-244f1b3edee9","Type":"ContainerStarted","Data":"94779940e5226e2780fabeb306e7336a8226012a45f404e88d95de1aefe0ed12"}
Sep 11 13:30:17.720433 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:17.720399    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46" event={"ID":"4c1679f3-5ebe-47de-948e-244f1b3edee9","Type":"ContainerStarted","Data":"c7137c16c3fe622b85470db65300c5d11198cdbc67136a99d4519e07728c25a3"}
Sep 11 13:30:17.720846 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:17.720479    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:21.218317 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:21.218283    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:21.218809 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:21.218348    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:21.230951 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:21.230921    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:21.250115 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:21.249996    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46" podStartSLOduration=6.747002953 podStartE2EDuration="7.249976793s" totalImagesPullingTime="502.97384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:17.744502793 +0000 UTC m=+1541.061959833" watchObservedRunningTime="2026-09-11 13:30:21.249976793 +0000 UTC m=+1544.567433833"
Sep 11 13:30:21.760988 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:21.760955    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:30:23.731345 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:23.731313    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:30:37.280752 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:37.280719    2554 scope.go:117] "RemoveContainer" containerID="0dc04153248f38488cfe3f86077504b01c7aaacaa1c881ea621203741570b675"
Sep 11 13:30:38.209853 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:38.209815    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:38.955509 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:38.955474    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:39.856986 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:39.856950    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:40.825709 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:40.825676    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:41.087659 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:41.087578    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:41.792450 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:41.792411    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:41.897565 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:41.897517    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:41.982780 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:41.982740    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:42.323995 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:42.323958    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:42.794663 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:42.794573    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:43.896882 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:43.896844    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:44.745655 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:44.745616    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:45.361662 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:45.361628    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:45.487294 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:45.487261    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:45.602659 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:45.602625    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:46.059923 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:46.059883    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:47.296204 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:47.296173    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:48.031439 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:48.031406    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:48.862944 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:48.862911    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:49.691425 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:49.691390    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:50.697487 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.697451    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"]
Sep 11 13:30:50.701422 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.701398    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:50.704022 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.703999    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-6c3a36-serving-cert\""
Sep 11 13:30:50.709503 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.709477    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-openshift-service-ca\") pod \"maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:50.709618 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.709543    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-maas-api-tls\") pod \"maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:50.709682 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.709659    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz427\" (UniqueName: \"kubernetes.io/projected/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-kube-api-access-cz427\") pod \"maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:50.709748 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.709734    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-metrics-certs\") pod \"maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:50.714842 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.714812    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"]
Sep 11 13:30:50.739165 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.739133    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:50.810686 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.810457    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz427\" (UniqueName: \"kubernetes.io/projected/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-kube-api-access-cz427\") pod \"maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:50.810686 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.810648    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-metrics-certs\") pod \"maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:50.810993 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.810708    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-openshift-service-ca\") pod \"maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:50.810993 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.810755    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-maas-api-tls\") pod \"maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:50.810993 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:30:50.810957    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-serving-cert: secret "maas-api-e2e-api-b-6c3a36-serving-cert" not found
Sep 11 13:30:50.811191 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:30:50.811081    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-maas-api-tls podName:b741fc75-71cf-44f0-9ea2-afaec1b36c5e nodeName:}" failed. No retries permitted until 2026-09-11 13:30:51.311055748 +0000 UTC m=+1574.628512786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-maas-api-tls") pod "maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc" (UID: "b741fc75-71cf-44f0-9ea2-afaec1b36c5e") : secret "maas-api-e2e-api-b-6c3a36-serving-cert" not found
Sep 11 13:30:50.812255 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.812230    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-openshift-service-ca\") pod \"maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:50.814108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.814087    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-metrics-certs\") pod \"maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:50.819617 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.819597    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz427\" (UniqueName: \"kubernetes.io/projected/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-kube-api-access-cz427\") pod \"maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:50.840267 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:50.840202    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:51.100780 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:51.100742    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:51.284718 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:51.284682    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:51.316591 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:51.316553    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-maas-api-tls\") pod \"maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:51.319287 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:51.319256    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-maas-api-tls\") pod \"maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:51.383796 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:51.383716    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:51.614050 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:51.614012    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:51.889965 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:51.889935    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"]
Sep 11 13:30:51.891409 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:30:51.891378    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb741fc75_71cf_44f0_9ea2_afaec1b36c5e.slice/crio-eed0d971c4db7734c9bad1589708f0e2207fdb5b50a3fcb275da550f06c9f1f4 WatchSource:0}: Error finding container eed0d971c4db7734c9bad1589708f0e2207fdb5b50a3fcb275da550f06c9f1f4: Status 404 returned error can't find the container with id eed0d971c4db7734c9bad1589708f0e2207fdb5b50a3fcb275da550f06c9f1f4
Sep 11 13:30:51.904283 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:51.904252    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc" event={"ID":"b741fc75-71cf-44f0-9ea2-afaec1b36c5e","Type":"ContainerStarted","Data":"eed0d971c4db7734c9bad1589708f0e2207fdb5b50a3fcb275da550f06c9f1f4"}
Sep 11 13:30:52.911599 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:52.911561    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc" event={"ID":"b741fc75-71cf-44f0-9ea2-afaec1b36c5e","Type":"ContainerStarted","Data":"15652223b028812f16403b04eef19a99f68816610d064d04e1e90a3985a18338"}
Sep 11 13:30:52.911984 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:52.911683    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:30:52.932395 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:52.932310    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc" podStartSLOduration=2.306043805 podStartE2EDuration="2.932289818s" totalImagesPullingTime="626.246013ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:52.931907863 +0000 UTC m=+1576.249364919" watchObservedRunningTime="2026-09-11 13:30:52.932289818 +0000 UTC m=+1576.249746859"
Sep 11 13:30:58.922409 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:30:58.922380    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:31:00.775564 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:00.775526    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:11.764959 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:11.764923    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:13.509695 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:13.509659    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:14.323123 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:14.323091    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:15.341790 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:15.341752    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:15.558517 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:15.558481    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:16.526334 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:16.526298    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:17.945024 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:17.944989    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:19.079472 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:19.079431    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:20.038501 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:20.038471    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:20.944358 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:20.944322    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:22.040846 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:22.040805    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:23.004812 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:23.004765    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:23.819921 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:23.819883    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:24.565914 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:24.565879    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:28.677789 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:28.677757    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:30.329709 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:30.329629    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:34.288385 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.288331    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"]
Sep 11 13:31:34.288822 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.288562    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46" podUID="4c1679f3-5ebe-47de-948e-244f1b3edee9" containerName="maas-api" containerID="cri-o://c7137c16c3fe622b85470db65300c5d11198cdbc67136a99d4519e07728c25a3" gracePeriod=30
Sep 11 13:31:34.313284 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.313251    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:34.421759 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.421713    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:34.521877 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.521848    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:34.534582 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.534560    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:31:34.600402 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.600347    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:34.620499 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.620462    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-metrics-certs\") pod \"4c1679f3-5ebe-47de-948e-244f1b3edee9\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") "
Sep 11 13:31:34.620673 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.620521    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlxxz\" (UniqueName: \"kubernetes.io/projected/4c1679f3-5ebe-47de-948e-244f1b3edee9-kube-api-access-rlxxz\") pod \"4c1679f3-5ebe-47de-948e-244f1b3edee9\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") "
Sep 11 13:31:34.620673 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.620542    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-maas-api-tls\") pod \"4c1679f3-5ebe-47de-948e-244f1b3edee9\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") "
Sep 11 13:31:34.620673 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.620609    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c1679f3-5ebe-47de-948e-244f1b3edee9-openshift-service-ca\") pod \"4c1679f3-5ebe-47de-948e-244f1b3edee9\" (UID: \"4c1679f3-5ebe-47de-948e-244f1b3edee9\") "
Sep 11 13:31:34.621048 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.621021    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c1679f3-5ebe-47de-948e-244f1b3edee9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c1679f3-5ebe-47de-948e-244f1b3edee9" (UID: "4c1679f3-5ebe-47de-948e-244f1b3edee9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:31:34.622649 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.622622    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4c1679f3-5ebe-47de-948e-244f1b3edee9" (UID: "4c1679f3-5ebe-47de-948e-244f1b3edee9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:31:34.622773 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.622668    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c1679f3-5ebe-47de-948e-244f1b3edee9" (UID: "4c1679f3-5ebe-47de-948e-244f1b3edee9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:31:34.622773 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.622681    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c1679f3-5ebe-47de-948e-244f1b3edee9-kube-api-access-rlxxz" (OuterVolumeSpecName: "kube-api-access-rlxxz") pod "4c1679f3-5ebe-47de-948e-244f1b3edee9" (UID: "4c1679f3-5ebe-47de-948e-244f1b3edee9"). InnerVolumeSpecName "kube-api-access-rlxxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:31:34.722264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.722222    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c1679f3-5ebe-47de-948e-244f1b3edee9-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:34.722264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.722257    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:34.722264 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.722271    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlxxz\" (UniqueName: \"kubernetes.io/projected/4c1679f3-5ebe-47de-948e-244f1b3edee9-kube-api-access-rlxxz\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:34.722536 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.722283    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c1679f3-5ebe-47de-948e-244f1b3edee9-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:34.938724 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:34.938641    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:35.064327 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:35.064290    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:35.131088 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:35.131048    2554 generic.go:356] "Generic (PLEG): container finished" podID="4c1679f3-5ebe-47de-948e-244f1b3edee9" containerID="c7137c16c3fe622b85470db65300c5d11198cdbc67136a99d4519e07728c25a3" exitCode=0
Sep 11 13:31:35.131088 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:35.131089    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"
Sep 11 13:31:35.131293 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:35.131142    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46" event={"ID":"4c1679f3-5ebe-47de-948e-244f1b3edee9","Type":"ContainerDied","Data":"c7137c16c3fe622b85470db65300c5d11198cdbc67136a99d4519e07728c25a3"}
Sep 11 13:31:35.131293 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:35.131191    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46" event={"ID":"4c1679f3-5ebe-47de-948e-244f1b3edee9","Type":"ContainerDied","Data":"94779940e5226e2780fabeb306e7336a8226012a45f404e88d95de1aefe0ed12"}
Sep 11 13:31:35.131293 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:35.131218    2554 scope.go:117] "RemoveContainer" containerID="c7137c16c3fe622b85470db65300c5d11198cdbc67136a99d4519e07728c25a3"
Sep 11 13:31:35.140750 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:35.140727    2554 scope.go:117] "RemoveContainer" containerID="c7137c16c3fe622b85470db65300c5d11198cdbc67136a99d4519e07728c25a3"
Sep 11 13:31:35.141057 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:31:35.141037    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7137c16c3fe622b85470db65300c5d11198cdbc67136a99d4519e07728c25a3\": container with ID starting with c7137c16c3fe622b85470db65300c5d11198cdbc67136a99d4519e07728c25a3 not found: ID does not exist" containerID="c7137c16c3fe622b85470db65300c5d11198cdbc67136a99d4519e07728c25a3"
Sep 11 13:31:35.141111 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:35.141064    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7137c16c3fe622b85470db65300c5d11198cdbc67136a99d4519e07728c25a3"} err="failed to get container status \"c7137c16c3fe622b85470db65300c5d11198cdbc67136a99d4519e07728c25a3\": rpc error: code = NotFound desc = could not find container \"c7137c16c3fe622b85470db65300c5d11198cdbc67136a99d4519e07728c25a3\": container with ID starting with c7137c16c3fe622b85470db65300c5d11198cdbc67136a99d4519e07728c25a3 not found: ID does not exist"
Sep 11 13:31:35.161425 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:35.161399    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"]
Sep 11 13:31:35.165515 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:35.165493    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0326b6-7b5574848d-qsf46"]
Sep 11 13:31:36.779834 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:36.779801    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:36.870840 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:36.870802    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:37.159083 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:37.159049    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:37.162484 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:37.162460    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c1679f3-5ebe-47de-948e-244f1b3edee9" path="/var/lib/kubelet/pods/4c1679f3-5ebe-47de-948e-244f1b3edee9/volumes"
Sep 11 13:31:37.930421 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:37.930387    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"]
Sep 11 13:31:37.949940 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:37.949903    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:38.049564 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:38.049521    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:38.939240 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:38.939208    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:39.658342 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:39.658308    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:41.562352 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:41.562319    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:41.636864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:41.636831    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:41.837527 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:41.837493    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:42.513435 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:42.513405    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:43.355773 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:43.355734    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:44.242634 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:44.242600    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:44.954597 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:44.954559    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:46.141658 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:46.141618    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:46.932045 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:46.932010    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:47.746357 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:47.746326    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:48.102262 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:48.102212    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:52.949464 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:52.949401    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql" podUID="ce62669f-837e-41fb-9387-fc6d60a633f0" containerName="main" containerID="cri-o://164265cc84cb282424d67643594509692d929e1bb595d21833fa45b96f3c78e3" gracePeriod=45
Sep 11 13:31:53.199621 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.199562    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:31:53.227654 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.227615    2554 generic.go:356] "Generic (PLEG): container finished" podID="ce62669f-837e-41fb-9387-fc6d60a633f0" containerID="164265cc84cb282424d67643594509692d929e1bb595d21833fa45b96f3c78e3" exitCode=0
Sep 11 13:31:53.227654 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.227649    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"
Sep 11 13:31:53.227892 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.227662    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql" event={"ID":"ce62669f-837e-41fb-9387-fc6d60a633f0","Type":"ContainerDied","Data":"164265cc84cb282424d67643594509692d929e1bb595d21833fa45b96f3c78e3"}
Sep 11 13:31:53.227892 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.227728    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql" event={"ID":"ce62669f-837e-41fb-9387-fc6d60a633f0","Type":"ContainerDied","Data":"68cf1df618c98f67411dfe4c3161c9b72133a42388c9c1b3cccc628785413d8e"}
Sep 11 13:31:53.227892 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.227751    2554 scope.go:117] "RemoveContainer" containerID="164265cc84cb282424d67643594509692d929e1bb595d21833fa45b96f3c78e3"
Sep 11 13:31:53.236106 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.236087    2554 scope.go:117] "RemoveContainer" containerID="164265cc84cb282424d67643594509692d929e1bb595d21833fa45b96f3c78e3"
Sep 11 13:31:53.236377 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:31:53.236343    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"164265cc84cb282424d67643594509692d929e1bb595d21833fa45b96f3c78e3\": container with ID starting with 164265cc84cb282424d67643594509692d929e1bb595d21833fa45b96f3c78e3 not found: ID does not exist" containerID="164265cc84cb282424d67643594509692d929e1bb595d21833fa45b96f3c78e3"
Sep 11 13:31:53.236493 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.236472    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"164265cc84cb282424d67643594509692d929e1bb595d21833fa45b96f3c78e3"} err="failed to get container status \"164265cc84cb282424d67643594509692d929e1bb595d21833fa45b96f3c78e3\": rpc error: code = NotFound desc = could not find container \"164265cc84cb282424d67643594509692d929e1bb595d21833fa45b96f3c78e3\": container with ID starting with 164265cc84cb282424d67643594509692d929e1bb595d21833fa45b96f3c78e3 not found: ID does not exist"
Sep 11 13:31:53.295404 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.295354    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-tmp-dir\") pod \"ce62669f-837e-41fb-9387-fc6d60a633f0\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") "
Sep 11 13:31:53.295557 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.295447    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbrds\" (UniqueName: \"kubernetes.io/projected/ce62669f-837e-41fb-9387-fc6d60a633f0-kube-api-access-qbrds\") pod \"ce62669f-837e-41fb-9387-fc6d60a633f0\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") "
Sep 11 13:31:53.295557 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.295467    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-home\") pod \"ce62669f-837e-41fb-9387-fc6d60a633f0\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") "
Sep 11 13:31:53.295557 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.295481    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-model-cache\") pod \"ce62669f-837e-41fb-9387-fc6d60a633f0\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") "
Sep 11 13:31:53.295557 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.295506    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce62669f-837e-41fb-9387-fc6d60a633f0-tls-certs\") pod \"ce62669f-837e-41fb-9387-fc6d60a633f0\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") "
Sep 11 13:31:53.295557 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.295555    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-dshm\") pod \"ce62669f-837e-41fb-9387-fc6d60a633f0\" (UID: \"ce62669f-837e-41fb-9387-fc6d60a633f0\") "
Sep 11 13:31:53.295808 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.295664    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce62669f-837e-41fb-9387-fc6d60a633f0" (UID: "ce62669f-837e-41fb-9387-fc6d60a633f0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:31:53.295808 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.295740    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-home" (OuterVolumeSpecName: "home") pod "ce62669f-837e-41fb-9387-fc6d60a633f0" (UID: "ce62669f-837e-41fb-9387-fc6d60a633f0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:31:53.295808 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.295761    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce62669f-837e-41fb-9387-fc6d60a633f0" (UID: "ce62669f-837e-41fb-9387-fc6d60a633f0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:31:53.295946 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.295863    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:53.295946 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.295882    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:53.295946 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.295895    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:53.297815 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.297780    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-dshm" (OuterVolumeSpecName: "dshm") pod "ce62669f-837e-41fb-9387-fc6d60a633f0" (UID: "ce62669f-837e-41fb-9387-fc6d60a633f0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:31:53.297910 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.297819    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce62669f-837e-41fb-9387-fc6d60a633f0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce62669f-837e-41fb-9387-fc6d60a633f0" (UID: "ce62669f-837e-41fb-9387-fc6d60a633f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:31:53.297910 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.297838    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce62669f-837e-41fb-9387-fc6d60a633f0-kube-api-access-qbrds" (OuterVolumeSpecName: "kube-api-access-qbrds") pod "ce62669f-837e-41fb-9387-fc6d60a633f0" (UID: "ce62669f-837e-41fb-9387-fc6d60a633f0"). InnerVolumeSpecName "kube-api-access-qbrds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:31:53.396613 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.396580    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbrds\" (UniqueName: \"kubernetes.io/projected/ce62669f-837e-41fb-9387-fc6d60a633f0-kube-api-access-qbrds\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:53.396613 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.396612    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce62669f-837e-41fb-9387-fc6d60a633f0-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:53.396828 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.396623    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce62669f-837e-41fb-9387-fc6d60a633f0-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:53.551035 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.550992    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"]
Sep 11 13:31:53.555261 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:53.555234    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/sub-test-model-12f386-kserve-bb76b6987-79hql"]
Sep 11 13:31:55.153266 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:55.153232    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce62669f-837e-41fb-9387-fc6d60a633f0" path="/var/lib/kubelet/pods/ce62669f-837e-41fb-9387-fc6d60a633f0/volumes"
Sep 11 13:31:57.739471 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:57.739437    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"]
Sep 11 13:31:57.739920 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:57.739651    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc" podUID="b741fc75-71cf-44f0-9ea2-afaec1b36c5e" containerName="maas-api" containerID="cri-o://15652223b028812f16403b04eef19a99f68816610d064d04e1e90a3985a18338" gracePeriod=30
Sep 11 13:31:57.771577 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:57.771548    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:57.848280 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:57.848244    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:57.929268 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:57.929232    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:57.987778 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:57.987749    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:31:58.015123 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.015041    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:58.092478 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.092443    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:58.140215 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.140186    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-openshift-service-ca\") pod \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") "
Sep 11 13:31:58.140215 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.140219    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-metrics-certs\") pod \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") "
Sep 11 13:31:58.140445 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.140259    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-maas-api-tls\") pod \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") "
Sep 11 13:31:58.140445 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.140372    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cz427\" (UniqueName: \"kubernetes.io/projected/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-kube-api-access-cz427\") pod \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\" (UID: \"b741fc75-71cf-44f0-9ea2-afaec1b36c5e\") "
Sep 11 13:31:58.140694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.140662    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b741fc75-71cf-44f0-9ea2-afaec1b36c5e" (UID: "b741fc75-71cf-44f0-9ea2-afaec1b36c5e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:31:58.142544 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.142514    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b741fc75-71cf-44f0-9ea2-afaec1b36c5e" (UID: "b741fc75-71cf-44f0-9ea2-afaec1b36c5e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:31:58.142660 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.142539    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b741fc75-71cf-44f0-9ea2-afaec1b36c5e" (UID: "b741fc75-71cf-44f0-9ea2-afaec1b36c5e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:31:58.142806 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.142782    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-kube-api-access-cz427" (OuterVolumeSpecName: "kube-api-access-cz427") pod "b741fc75-71cf-44f0-9ea2-afaec1b36c5e" (UID: "b741fc75-71cf-44f0-9ea2-afaec1b36c5e"). InnerVolumeSpecName "kube-api-access-cz427". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:31:58.241281 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.241243    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:58.241281 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.241272    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cz427\" (UniqueName: \"kubernetes.io/projected/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-kube-api-access-cz427\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:58.241281 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.241284    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:58.241611 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.241296    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b741fc75-71cf-44f0-9ea2-afaec1b36c5e-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:31:58.253480 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.253448    2554 generic.go:356] "Generic (PLEG): container finished" podID="b741fc75-71cf-44f0-9ea2-afaec1b36c5e" containerID="15652223b028812f16403b04eef19a99f68816610d064d04e1e90a3985a18338" exitCode=0
Sep 11 13:31:58.253613 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.253495    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"
Sep 11 13:31:58.253613 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.253535    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc" event={"ID":"b741fc75-71cf-44f0-9ea2-afaec1b36c5e","Type":"ContainerDied","Data":"15652223b028812f16403b04eef19a99f68816610d064d04e1e90a3985a18338"}
Sep 11 13:31:58.253613 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.253602    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc" event={"ID":"b741fc75-71cf-44f0-9ea2-afaec1b36c5e","Type":"ContainerDied","Data":"eed0d971c4db7734c9bad1589708f0e2207fdb5b50a3fcb275da550f06c9f1f4"}
Sep 11 13:31:58.253772 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.253633    2554 scope.go:117] "RemoveContainer" containerID="15652223b028812f16403b04eef19a99f68816610d064d04e1e90a3985a18338"
Sep 11 13:31:58.263044 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.263024    2554 scope.go:117] "RemoveContainer" containerID="15652223b028812f16403b04eef19a99f68816610d064d04e1e90a3985a18338"
Sep 11 13:31:58.263342 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:31:58.263323    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15652223b028812f16403b04eef19a99f68816610d064d04e1e90a3985a18338\": container with ID starting with 15652223b028812f16403b04eef19a99f68816610d064d04e1e90a3985a18338 not found: ID does not exist" containerID="15652223b028812f16403b04eef19a99f68816610d064d04e1e90a3985a18338"
Sep 11 13:31:58.263421 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.263351    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15652223b028812f16403b04eef19a99f68816610d064d04e1e90a3985a18338"} err="failed to get container status \"15652223b028812f16403b04eef19a99f68816610d064d04e1e90a3985a18338\": rpc error: code = NotFound desc = could not find container \"15652223b028812f16403b04eef19a99f68816610d064d04e1e90a3985a18338\": container with ID starting with 15652223b028812f16403b04eef19a99f68816610d064d04e1e90a3985a18338 not found: ID does not exist"
Sep 11 13:31:58.276723 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.276655    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"]
Sep 11 13:31:58.278640 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.278621    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6c3a36-cbfccf4f6-pk5sc"]
Sep 11 13:31:58.354714 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.354681    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:58.461207 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:58.461174    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:59.153973 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:59.153940    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b741fc75-71cf-44f0-9ea2-afaec1b36c5e" path="/var/lib/kubelet/pods/b741fc75-71cf-44f0-9ea2-afaec1b36c5e/volumes"
Sep 11 13:31:59.200066 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:59.200023    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:59.887432 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:31:59.887401    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:00.837471 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:00.837434    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:01.568464 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:01.568431    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:04.705336 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:04.705302    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:04.805673 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:04.805637    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:04.976442 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:04.976327    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:05.686200 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:05.686123    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:06.919589 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:06.919556    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:08.019204 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:08.019166    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:09.120451 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:09.120415    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:10.050161 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:10.050122    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:10.804674 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:10.804641    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:16.096640 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.096604    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"]
Sep 11 13:32:16.097113 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.096832    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd" podUID="c1cf9568-e5f7-462c-b6f6-e35f343b8b2c" containerName="maas-api" containerID="cri-o://f58ddd8e519c38d234b67d7b5d9fd3018d76b93d3061a0d192f0ad38cdbdf911" gracePeriod=30
Sep 11 13:32:16.156678 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.156631    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:16.331440 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.331358    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:16.348551 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.348522    2554 generic.go:356] "Generic (PLEG): container finished" podID="c1cf9568-e5f7-462c-b6f6-e35f343b8b2c" containerID="f58ddd8e519c38d234b67d7b5d9fd3018d76b93d3061a0d192f0ad38cdbdf911" exitCode=0
Sep 11 13:32:16.348660 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.348560    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd" event={"ID":"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c","Type":"ContainerDied","Data":"f58ddd8e519c38d234b67d7b5d9fd3018d76b93d3061a0d192f0ad38cdbdf911"}
Sep 11 13:32:16.362943 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.362911    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:32:16.402842 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.402739    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-metrics-certs\") pod \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") "
Sep 11 13:32:16.402842 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.402782    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prc99\" (UniqueName: \"kubernetes.io/projected/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-kube-api-access-prc99\") pod \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") "
Sep 11 13:32:16.402842 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.402808    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-openshift-service-ca\") pod \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") "
Sep 11 13:32:16.403119 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.402884    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-maas-api-tls\") pod \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\" (UID: \"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c\") "
Sep 11 13:32:16.403472 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.403440    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c1cf9568-e5f7-462c-b6f6-e35f343b8b2c" (UID: "c1cf9568-e5f7-462c-b6f6-e35f343b8b2c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:32:16.405024 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.404996    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c1cf9568-e5f7-462c-b6f6-e35f343b8b2c" (UID: "c1cf9568-e5f7-462c-b6f6-e35f343b8b2c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:32:16.405129 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.405071    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c1cf9568-e5f7-462c-b6f6-e35f343b8b2c" (UID: "c1cf9568-e5f7-462c-b6f6-e35f343b8b2c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:32:16.405129 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.405108    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-kube-api-access-prc99" (OuterVolumeSpecName: "kube-api-access-prc99") pod "c1cf9568-e5f7-462c-b6f6-e35f343b8b2c" (UID: "c1cf9568-e5f7-462c-b6f6-e35f343b8b2c"). InnerVolumeSpecName "kube-api-access-prc99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:32:16.444617 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.444585    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:16.503747 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.503713    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:32:16.503747 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.503746    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:32:16.503970 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.503760    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prc99\" (UniqueName: \"kubernetes.io/projected/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-kube-api-access-prc99\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:32:16.503970 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.503773    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:32:16.587645 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.587614    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:16.674448 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.674310    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:16.927893 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:16.927783    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:17.174003 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:17.173968    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:17.355759 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:17.355732    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"
Sep 11 13:32:17.355928 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:17.355732    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd" event={"ID":"c1cf9568-e5f7-462c-b6f6-e35f343b8b2c","Type":"ContainerDied","Data":"60556bfd2f506cf7c6dab9ac9a3608aced04a748c2ca74047e8327b5f9f8f76e"}
Sep 11 13:32:17.355928 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:17.355873    2554 scope.go:117] "RemoveContainer" containerID="f58ddd8e519c38d234b67d7b5d9fd3018d76b93d3061a0d192f0ad38cdbdf911"
Sep 11 13:32:17.403472 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:17.403440    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"]
Sep 11 13:32:17.428435 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:17.428401    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-e0ebd7-6b5d9fb787-qclpd"]
Sep 11 13:32:18.336522 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:18.336482    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:19.153002 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:19.152968    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1cf9568-e5f7-462c-b6f6-e35f343b8b2c" path="/var/lib/kubelet/pods/c1cf9568-e5f7-462c-b6f6-e35f343b8b2c/volumes"
Sep 11 13:32:19.244163 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:19.244131    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:20.443005 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:20.442972    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:20.541007 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:20.540970    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:23.633959 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:23.633905    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:23.731477 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:23.731429    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:23.889382 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:23.889276    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:25.036327 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:25.036278    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:26.110122 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:26.110088    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:26.857939 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:26.857900    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:27.576265 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:27.576233    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:32.807467 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:32.807431    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:33.078561 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:33.078528    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:34.999540 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:34.999503    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:35.091456 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:35.091426    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:35.190299 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:35.190264    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:35.287771 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:35.287685    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:35.373457 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:35.373419    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:35.666788 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:35.666755    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:35.767287 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:35.767255    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:36.692510 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:36.692474    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:37.558391 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:37.558336    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:42.169002 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:42.168952    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:42.302912 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:42.302878    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:42.494673 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:32:42.494577    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:02.799052 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:02.798968    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:03.923065 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:03.923029    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:04.365710 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:04.365677    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:04.876247 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:04.876210    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:05.242293 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:05.242210    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:06.746475 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:06.746433    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:07.648341 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:07.648307    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:07.729612 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:07.729563    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:08.548832 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:08.548792    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:09.508335 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.508300    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"]
Sep 11 13:33:09.508837 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.508813    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b741fc75-71cf-44f0-9ea2-afaec1b36c5e" containerName="maas-api"
Sep 11 13:33:09.509004 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.508982    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b741fc75-71cf-44f0-9ea2-afaec1b36c5e" containerName="maas-api"
Sep 11 13:33:09.509004 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.509016    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1cf9568-e5f7-462c-b6f6-e35f343b8b2c" containerName="maas-api"
Sep 11 13:33:09.509187 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.509024    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1cf9568-e5f7-462c-b6f6-e35f343b8b2c" containerName="maas-api"
Sep 11 13:33:09.509187 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.509036    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c1679f3-5ebe-47de-948e-244f1b3edee9" containerName="maas-api"
Sep 11 13:33:09.509187 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.509045    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c1679f3-5ebe-47de-948e-244f1b3edee9" containerName="maas-api"
Sep 11 13:33:09.509187 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.509072    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce62669f-837e-41fb-9387-fc6d60a633f0" containerName="main"
Sep 11 13:33:09.509187 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.509080    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce62669f-837e-41fb-9387-fc6d60a633f0" containerName="main"
Sep 11 13:33:09.509476 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.509193    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce62669f-837e-41fb-9387-fc6d60a633f0" containerName="main"
Sep 11 13:33:09.509476 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.509208    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1cf9568-e5f7-462c-b6f6-e35f343b8b2c" containerName="maas-api"
Sep 11 13:33:09.509476 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.509216    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c1679f3-5ebe-47de-948e-244f1b3edee9" containerName="maas-api"
Sep 11 13:33:09.509476 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.509223    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b741fc75-71cf-44f0-9ea2-afaec1b36c5e" containerName="maas-api"
Sep 11 13:33:09.512124 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.512106    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:09.515459 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.515437    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-48243b-serving-cert\""
Sep 11 13:33:09.524041 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.524015    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"]
Sep 11 13:33:09.543707 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.543670    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:09.561835 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.561796    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67pw5\" (UniqueName: \"kubernetes.io/projected/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-kube-api-access-67pw5\") pod \"maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:09.562239 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.561859    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-metrics-certs\") pod \"maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:09.562239 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.561887    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:09.562239 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.561989    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:09.644561 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.644525    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:09.662684 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.662646    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67pw5\" (UniqueName: \"kubernetes.io/projected/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-kube-api-access-67pw5\") pod \"maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:09.662853 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.662694    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-metrics-certs\") pod \"maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:09.662853 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.662733    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:09.662853 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.662798    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:09.663511 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.663486    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:09.665239 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.665209    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:09.665334 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.665262    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-metrics-certs\") pod \"maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:09.672211 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.672190    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67pw5\" (UniqueName: \"kubernetes.io/projected/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-kube-api-access-67pw5\") pod \"maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:09.711146 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.711108    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:09.825432 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:09.825401    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:10.100564 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:10.100431    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"]
Sep 11 13:33:10.104009 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:33:10.103969    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d24189_1e24_4de8_a86e_6fdff2dcf67e.slice/crio-90eb47207137f7929a3166abd44e9c4b2d2b60c3b103dcdb57e9db12aadc470d WatchSource:0}: Error finding container 90eb47207137f7929a3166abd44e9c4b2d2b60c3b103dcdb57e9db12aadc470d: Status 404 returned error can't find the container with id 90eb47207137f7929a3166abd44e9c4b2d2b60c3b103dcdb57e9db12aadc470d
Sep 11 13:33:10.139837 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:10.139809    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:10.253055 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:10.253018    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:10.621528 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:10.621499    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng" event={"ID":"e2d24189-1e24-4de8-a86e-6fdff2dcf67e","Type":"ContainerStarted","Data":"90eb47207137f7929a3166abd44e9c4b2d2b60c3b103dcdb57e9db12aadc470d"}
Sep 11 13:33:11.627827 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:11.627790    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng" event={"ID":"e2d24189-1e24-4de8-a86e-6fdff2dcf67e","Type":"ContainerStarted","Data":"41629ac98e14ea7a6718b218f3ee4e78c844fe73bdb65330e1a6bb5597431566"}
Sep 11 13:33:11.628276 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:11.627874    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:11.648703 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:11.648643    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng" podStartSLOduration=2.133246716 podStartE2EDuration="2.648627464s" totalImagesPullingTime="515.380748ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:33:11.645951821 +0000 UTC m=+1714.963408887" watchObservedRunningTime="2026-09-11 13:33:11.648627464 +0000 UTC m=+1714.966084527"
Sep 11 13:33:17.639756 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:17.639728    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:33:31.667520 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:31.667485    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:32.378610 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:32.378557    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:32.532420 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:32.532353    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:33.032107 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:33.032071    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:33.347815 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:33.347781    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:33.446633 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:33.446598    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:34.024167 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:34.024132    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:34.251986 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:34.251930    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:35.919574 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:35.919534    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:36.654445 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:36.654408    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:36.752027 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:36.751996    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:37.729601 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:37.729560    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:38.490185 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:38.490147    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:39.118005 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.117959    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"]
Sep 11 13:33:39.121723 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.121701    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.124035 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.124011    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-47b957-serving-cert\""
Sep 11 13:33:39.132568 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.132537    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"]
Sep 11 13:33:39.164415 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.164381    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:39.211281 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.211247    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aac145f9-c8c8-496e-93e7-d3a03cce9049-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.211281 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.211287    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aac145f9-c8c8-496e-93e7-d3a03cce9049-metrics-certs\") pod \"maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.211522 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.211312    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mgxv\" (UniqueName: \"kubernetes.io/projected/aac145f9-c8c8-496e-93e7-d3a03cce9049-kube-api-access-6mgxv\") pod \"maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.211522 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.211333    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aac145f9-c8c8-496e-93e7-d3a03cce9049-maas-api-tls\") pod \"maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.262804 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.262767    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:39.312861 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.312824    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aac145f9-c8c8-496e-93e7-d3a03cce9049-metrics-certs\") pod \"maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.313040 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.312878    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mgxv\" (UniqueName: \"kubernetes.io/projected/aac145f9-c8c8-496e-93e7-d3a03cce9049-kube-api-access-6mgxv\") pod \"maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.313040 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.312903    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aac145f9-c8c8-496e-93e7-d3a03cce9049-maas-api-tls\") pod \"maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.313040 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.312979    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aac145f9-c8c8-496e-93e7-d3a03cce9049-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.313714 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.313691    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aac145f9-c8c8-496e-93e7-d3a03cce9049-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.315424 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.315405    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aac145f9-c8c8-496e-93e7-d3a03cce9049-maas-api-tls\") pod \"maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.315565 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.315544    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aac145f9-c8c8-496e-93e7-d3a03cce9049-metrics-certs\") pod \"maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.320863 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.320838    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mgxv\" (UniqueName: \"kubernetes.io/projected/aac145f9-c8c8-496e-93e7-d3a03cce9049-kube-api-access-6mgxv\") pod \"maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.326349 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.326319    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:39.433920 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.433817    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:39.620522 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.620480    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:39.716206 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.716136    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"]
Sep 11 13:33:39.718535 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.718493    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:39.718663 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:33:39.718646    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaac145f9_c8c8_496e_93e7_d3a03cce9049.slice/crio-9f2541d4d7adccf79cbe4774e3d7e0049df8e6c034312c759f7a29f9ccfb5a1c WatchSource:0}: Error finding container 9f2541d4d7adccf79cbe4774e3d7e0049df8e6c034312c759f7a29f9ccfb5a1c: Status 404 returned error can't find the container with id 9f2541d4d7adccf79cbe4774e3d7e0049df8e6c034312c759f7a29f9ccfb5a1c
Sep 11 13:33:39.769305 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:39.769273    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj" event={"ID":"aac145f9-c8c8-496e-93e7-d3a03cce9049","Type":"ContainerStarted","Data":"9f2541d4d7adccf79cbe4774e3d7e0049df8e6c034312c759f7a29f9ccfb5a1c"}
Sep 11 13:33:40.776396 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:40.776348    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj" event={"ID":"aac145f9-c8c8-496e-93e7-d3a03cce9049","Type":"ContainerStarted","Data":"7e53486b689180a607999c65b151c26c6dec57fefdbe0ae9f9878afaca14954b"}
Sep 11 13:33:40.776815 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:40.776418    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:40.796137 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:40.796086    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj" podStartSLOduration=1.399983136 podStartE2EDuration="1.796069651s" totalImagesPullingTime="396.086515ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:33:40.795008136 +0000 UTC m=+1744.112465176" watchObservedRunningTime="2026-09-11 13:33:40.796069651 +0000 UTC m=+1744.113526690"
Sep 11 13:33:45.592419 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:45.592387    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:46.787269 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:46.787236    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:33:52.356674 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:52.356642    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:55.609256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:55.609219    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:55.673979 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:33:55.673943    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:01.423421 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:34:01.423382    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:01.592004 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:34:01.591965    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:02.130738 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:34:02.130706    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:02.953307 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:34:02.953270    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:03.679520 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:34:03.679486    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:04.392892 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:34:04.392862    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:33.000178 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:34:33.000132    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:37.223883 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:34:37.223848    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:34:37.227721 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:34:37.227698    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:35:13.609571 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:13.609529    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:15.119545 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:15.119507    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:16.378352 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:16.378311    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:17.351652 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:17.351618    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:17.772224 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:17.772126    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:18.593943 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:18.593882    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:20.498807 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:20.498773    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:21.943527 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:21.943491    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:23.090965 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:23.090929    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:24.075525 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:24.075482    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:24.192428 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:24.192394    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:24.539869 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:24.539785    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:24.638967 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:24.638934    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:24.740202 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:24.740163    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:45.013661 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:45.013620    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:46.213292 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:46.213259    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:48.150283 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:48.150220    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:48.861997 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:48.861964    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:49.807522 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:49.807490    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:50.958566 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:50.958534    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:51.745758 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:51.745722    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:52.394664 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:52.394629    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:52.903348 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:52.903313    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:54.587407 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:54.587355    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:55.863406 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:55.863343    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:56.808409 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:56.808357    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:57.643381 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:57.643332    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:58.581545 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:58.581508    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:59.324992 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:35:59.324948    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:04.109588 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:04.109548    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:04.259945 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:04.259910    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:04.393907 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:04.393820    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:04.523706 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:04.523675    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:04.951940 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:04.951902    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:05.065679 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:05.065646    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:05.975552 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:05.975509    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:06.837915 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:06.837878    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:07.949513 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:07.949478    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:08.761667 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:08.761622    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:11.290025 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:11.289984    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:11.405844 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:11.405808    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:11.599542 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:11.599500    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:16.105248 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:16.105215    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:17.123330 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:17.123282    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:17.977193 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:17.977154    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:18.887271 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:18.887232    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:19.634268 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:19.634232    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:24.623520 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:24.623488    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:30.282160 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.282122    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"]
Sep 11 13:36:30.282601 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.282323    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng" podUID="e2d24189-1e24-4de8-a86e-6fdff2dcf67e" containerName="maas-api" containerID="cri-o://41629ac98e14ea7a6718b218f3ee4e78c844fe73bdb65330e1a6bb5597431566" gracePeriod=30
Sep 11 13:36:30.323040 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.323006    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:30.412306 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.412266    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:30.517933 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.517905    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:30.530401 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.530350    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:36:30.617935 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.617900    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:30.630088 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.630054    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-maas-api-tls\") pod \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") "
Sep 11 13:36:30.630259 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.630131    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-openshift-service-ca\") pod \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") "
Sep 11 13:36:30.630259 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.630176    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67pw5\" (UniqueName: \"kubernetes.io/projected/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-kube-api-access-67pw5\") pod \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") "
Sep 11 13:36:30.630259 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.630215    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-metrics-certs\") pod \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\" (UID: \"e2d24189-1e24-4de8-a86e-6fdff2dcf67e\") "
Sep 11 13:36:30.630757 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.630725    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e2d24189-1e24-4de8-a86e-6fdff2dcf67e" (UID: "e2d24189-1e24-4de8-a86e-6fdff2dcf67e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:36:30.632675 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.632643    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e2d24189-1e24-4de8-a86e-6fdff2dcf67e" (UID: "e2d24189-1e24-4de8-a86e-6fdff2dcf67e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:36:30.633229 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.633169    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-kube-api-access-67pw5" (OuterVolumeSpecName: "kube-api-access-67pw5") pod "e2d24189-1e24-4de8-a86e-6fdff2dcf67e" (UID: "e2d24189-1e24-4de8-a86e-6fdff2dcf67e"). InnerVolumeSpecName "kube-api-access-67pw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:36:30.633229 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.633201    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e2d24189-1e24-4de8-a86e-6fdff2dcf67e" (UID: "e2d24189-1e24-4de8-a86e-6fdff2dcf67e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:36:30.657313 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.657278    2554 generic.go:356] "Generic (PLEG): container finished" podID="e2d24189-1e24-4de8-a86e-6fdff2dcf67e" containerID="41629ac98e14ea7a6718b218f3ee4e78c844fe73bdb65330e1a6bb5597431566" exitCode=0
Sep 11 13:36:30.657477 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.657328    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"
Sep 11 13:36:30.657477 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.657358    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng" event={"ID":"e2d24189-1e24-4de8-a86e-6fdff2dcf67e","Type":"ContainerDied","Data":"41629ac98e14ea7a6718b218f3ee4e78c844fe73bdb65330e1a6bb5597431566"}
Sep 11 13:36:30.657477 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.657440    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng" event={"ID":"e2d24189-1e24-4de8-a86e-6fdff2dcf67e","Type":"ContainerDied","Data":"90eb47207137f7929a3166abd44e9c4b2d2b60c3b103dcdb57e9db12aadc470d"}
Sep 11 13:36:30.657477 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.657472    2554 scope.go:117] "RemoveContainer" containerID="41629ac98e14ea7a6718b218f3ee4e78c844fe73bdb65330e1a6bb5597431566"
Sep 11 13:36:30.671409 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.671390    2554 scope.go:117] "RemoveContainer" containerID="41629ac98e14ea7a6718b218f3ee4e78c844fe73bdb65330e1a6bb5597431566"
Sep 11 13:36:30.671758 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:36:30.671737    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41629ac98e14ea7a6718b218f3ee4e78c844fe73bdb65330e1a6bb5597431566\": container with ID starting with 41629ac98e14ea7a6718b218f3ee4e78c844fe73bdb65330e1a6bb5597431566 not found: ID does not exist" containerID="41629ac98e14ea7a6718b218f3ee4e78c844fe73bdb65330e1a6bb5597431566"
Sep 11 13:36:30.671801 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.671763    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41629ac98e14ea7a6718b218f3ee4e78c844fe73bdb65330e1a6bb5597431566"} err="failed to get container status \"41629ac98e14ea7a6718b218f3ee4e78c844fe73bdb65330e1a6bb5597431566\": rpc error: code = NotFound desc = could not find container \"41629ac98e14ea7a6718b218f3ee4e78c844fe73bdb65330e1a6bb5597431566\": container with ID starting with 41629ac98e14ea7a6718b218f3ee4e78c844fe73bdb65330e1a6bb5597431566 not found: ID does not exist"
Sep 11 13:36:30.685279 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.685242    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"]
Sep 11 13:36:30.689665 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.689628    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-48243b-794dc7dfc-qfsng"]
Sep 11 13:36:30.731512 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.731470    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:36:30.731512 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.731504    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:36:30.731512 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.731520    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67pw5\" (UniqueName: \"kubernetes.io/projected/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-kube-api-access-67pw5\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:36:30.731751 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.731534    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d24189-1e24-4de8-a86e-6fdff2dcf67e-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:36:30.908178 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:30.908064    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:31.021125 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:31.021081    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:31.153508 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:31.153472    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2d24189-1e24-4de8-a86e-6fdff2dcf67e" path="/var/lib/kubelet/pods/e2d24189-1e24-4de8-a86e-6fdff2dcf67e/volumes"
Sep 11 13:36:31.751537 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:31.751501    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:32.934328 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:32.934292    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:33.801373 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:33.801335    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:36.017642 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:36:36.017608    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:12.822094 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:12.822055    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:13.465601 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:13.465562    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:19.694641 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:19.694604    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:19.800600 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:19.800564    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:20.044423 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:20.044304    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:20.693085 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:20.693050    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:21.629070 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:21.629031    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:22.593822 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:22.593781    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:23.346014 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:23.345973    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:24.116635 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:24.116587    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:31.038605 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.038571    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"]
Sep 11 13:37:31.039084 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.038775    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj" podUID="aac145f9-c8c8-496e-93e7-d3a03cce9049" containerName="maas-api" containerID="cri-o://7e53486b689180a607999c65b151c26c6dec57fefdbe0ae9f9878afaca14954b" gracePeriod=30
Sep 11 13:37:31.066198 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.066152    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:31.149302 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.149187    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:31.247987 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.247948    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:31.297803 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.297774    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:37:31.315570 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.315542    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:31.385812 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.385771    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aac145f9-c8c8-496e-93e7-d3a03cce9049-metrics-certs\") pod \"aac145f9-c8c8-496e-93e7-d3a03cce9049\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") "
Sep 11 13:37:31.385812 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.385815    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mgxv\" (UniqueName: \"kubernetes.io/projected/aac145f9-c8c8-496e-93e7-d3a03cce9049-kube-api-access-6mgxv\") pod \"aac145f9-c8c8-496e-93e7-d3a03cce9049\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") "
Sep 11 13:37:31.386086 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.385872    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aac145f9-c8c8-496e-93e7-d3a03cce9049-maas-api-tls\") pod \"aac145f9-c8c8-496e-93e7-d3a03cce9049\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") "
Sep 11 13:37:31.386086 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.385974    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aac145f9-c8c8-496e-93e7-d3a03cce9049-openshift-service-ca\") pod \"aac145f9-c8c8-496e-93e7-d3a03cce9049\" (UID: \"aac145f9-c8c8-496e-93e7-d3a03cce9049\") "
Sep 11 13:37:31.386398 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.386346    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aac145f9-c8c8-496e-93e7-d3a03cce9049-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aac145f9-c8c8-496e-93e7-d3a03cce9049" (UID: "aac145f9-c8c8-496e-93e7-d3a03cce9049"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:31.387966 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.387944    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aac145f9-c8c8-496e-93e7-d3a03cce9049-kube-api-access-6mgxv" (OuterVolumeSpecName: "kube-api-access-6mgxv") pod "aac145f9-c8c8-496e-93e7-d3a03cce9049" (UID: "aac145f9-c8c8-496e-93e7-d3a03cce9049"). InnerVolumeSpecName "kube-api-access-6mgxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:31.387966 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.387950    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aac145f9-c8c8-496e-93e7-d3a03cce9049-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aac145f9-c8c8-496e-93e7-d3a03cce9049" (UID: "aac145f9-c8c8-496e-93e7-d3a03cce9049"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:37:31.388103 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.387964    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aac145f9-c8c8-496e-93e7-d3a03cce9049-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aac145f9-c8c8-496e-93e7-d3a03cce9049" (UID: "aac145f9-c8c8-496e-93e7-d3a03cce9049"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:37:31.487033 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.486948    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aac145f9-c8c8-496e-93e7-d3a03cce9049-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:37:31.487033 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.486979    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aac145f9-c8c8-496e-93e7-d3a03cce9049-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:37:31.487033 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.486988    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aac145f9-c8c8-496e-93e7-d3a03cce9049-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:37:31.487033 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.486998    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mgxv\" (UniqueName: \"kubernetes.io/projected/aac145f9-c8c8-496e-93e7-d3a03cce9049-kube-api-access-6mgxv\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:37:31.713073 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.713037    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:31.843191 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.843150    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:31.977922 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.977891    2554 generic.go:356] "Generic (PLEG): container finished" podID="aac145f9-c8c8-496e-93e7-d3a03cce9049" containerID="7e53486b689180a607999c65b151c26c6dec57fefdbe0ae9f9878afaca14954b" exitCode=0
Sep 11 13:37:31.977922 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.977923    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"
Sep 11 13:37:31.978141 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.977960    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj" event={"ID":"aac145f9-c8c8-496e-93e7-d3a03cce9049","Type":"ContainerDied","Data":"7e53486b689180a607999c65b151c26c6dec57fefdbe0ae9f9878afaca14954b"}
Sep 11 13:37:31.978141 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.978012    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj" event={"ID":"aac145f9-c8c8-496e-93e7-d3a03cce9049","Type":"ContainerDied","Data":"9f2541d4d7adccf79cbe4774e3d7e0049df8e6c034312c759f7a29f9ccfb5a1c"}
Sep 11 13:37:31.978141 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.978035    2554 scope.go:117] "RemoveContainer" containerID="7e53486b689180a607999c65b151c26c6dec57fefdbe0ae9f9878afaca14954b"
Sep 11 13:37:31.987742 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.987722    2554 scope.go:117] "RemoveContainer" containerID="7e53486b689180a607999c65b151c26c6dec57fefdbe0ae9f9878afaca14954b"
Sep 11 13:37:31.988028 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:37:31.988012    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e53486b689180a607999c65b151c26c6dec57fefdbe0ae9f9878afaca14954b\": container with ID starting with 7e53486b689180a607999c65b151c26c6dec57fefdbe0ae9f9878afaca14954b not found: ID does not exist" containerID="7e53486b689180a607999c65b151c26c6dec57fefdbe0ae9f9878afaca14954b"
Sep 11 13:37:31.988068 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:31.988037    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e53486b689180a607999c65b151c26c6dec57fefdbe0ae9f9878afaca14954b"} err="failed to get container status \"7e53486b689180a607999c65b151c26c6dec57fefdbe0ae9f9878afaca14954b\": rpc error: code = NotFound desc = could not find container \"7e53486b689180a607999c65b151c26c6dec57fefdbe0ae9f9878afaca14954b\": container with ID starting with 7e53486b689180a607999c65b151c26c6dec57fefdbe0ae9f9878afaca14954b not found: ID does not exist"
Sep 11 13:37:32.012477 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:32.012440    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"]
Sep 11 13:37:32.025236 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:32.025198    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-47b957-7d8947df8f-r8qdj"]
Sep 11 13:37:32.747063 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:32.747028    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:33.153428 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:33.153397    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aac145f9-c8c8-496e-93e7-d3a03cce9049" path="/var/lib/kubelet/pods/aac145f9-c8c8-496e-93e7-d3a03cce9049/volumes"
Sep 11 13:37:33.617077 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:33.617041    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:38.074051 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:38.074007    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:38.147767 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:38.147725    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:38.345650 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:38.345562    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:38.927534 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:38.927497    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:39.765626 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:39.765588    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:40.759245 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:40.759209    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:41.511973 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:41.511930    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:48.950534 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:48.950488    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:49.329067 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.329033    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"]
Sep 11 13:37:49.329262 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.329241    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x" podUID="30e0f013-8b18-42f9-94fe-6cdf5cc7b71b" containerName="maas-api" containerID="cri-o://f139e783431b1614f6a5c516d619f38ae07eb5ecdb48ddff40cde90e6d7bb1e9" gracePeriod=30
Sep 11 13:37:49.362648 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.362608    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:49.368197 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.368167    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6ffb6a-29818890-4mh2j"]
Sep 11 13:37:49.373175 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.373149    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-6ffb6a-29818890-4mh2j"]
Sep 11 13:37:49.464527 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.464490    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:49.545665 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.545633    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:49.582181 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.582118    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:37:49.648779 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.648678    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:49.734479 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.734449    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:49.746548 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.746521    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-metrics-certs\") pod \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") "
Sep 11 13:37:49.746686 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.746555    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-maas-api-tls\") pod \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") "
Sep 11 13:37:49.746686 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.746602    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-openshift-service-ca\") pod \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") "
Sep 11 13:37:49.746765 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.746702    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g48dh\" (UniqueName: \"kubernetes.io/projected/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-kube-api-access-g48dh\") pod \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\" (UID: \"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b\") "
Sep 11 13:37:49.746988 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.746956    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "30e0f013-8b18-42f9-94fe-6cdf5cc7b71b" (UID: "30e0f013-8b18-42f9-94fe-6cdf5cc7b71b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:49.748851 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.748816    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-kube-api-access-g48dh" (OuterVolumeSpecName: "kube-api-access-g48dh") pod "30e0f013-8b18-42f9-94fe-6cdf5cc7b71b" (UID: "30e0f013-8b18-42f9-94fe-6cdf5cc7b71b"). InnerVolumeSpecName "kube-api-access-g48dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:49.748968 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.748880    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "30e0f013-8b18-42f9-94fe-6cdf5cc7b71b" (UID: "30e0f013-8b18-42f9-94fe-6cdf5cc7b71b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:37:49.748968 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.748895    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "30e0f013-8b18-42f9-94fe-6cdf5cc7b71b" (UID: "30e0f013-8b18-42f9-94fe-6cdf5cc7b71b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:37:49.847819 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.847727    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g48dh\" (UniqueName: \"kubernetes.io/projected/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-kube-api-access-g48dh\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:37:49.847819 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.847760    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:37:49.847819 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.847769    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:37:49.847819 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:49.847776    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:37:50.043585 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:50.043544    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:50.068958 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:50.068918    2554 generic.go:356] "Generic (PLEG): container finished" podID="30e0f013-8b18-42f9-94fe-6cdf5cc7b71b" containerID="f139e783431b1614f6a5c516d619f38ae07eb5ecdb48ddff40cde90e6d7bb1e9" exitCode=0
Sep 11 13:37:50.069122 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:50.068981    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"
Sep 11 13:37:50.069122 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:50.069002    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x" event={"ID":"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b","Type":"ContainerDied","Data":"f139e783431b1614f6a5c516d619f38ae07eb5ecdb48ddff40cde90e6d7bb1e9"}
Sep 11 13:37:50.069122 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:50.069085    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x" event={"ID":"30e0f013-8b18-42f9-94fe-6cdf5cc7b71b","Type":"ContainerDied","Data":"3ef75f3ea2b5cf1c54400d752b3d2907604d5a9a764a46a88b0b8434da6704cd"}
Sep 11 13:37:50.069122 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:50.069108    2554 scope.go:117] "RemoveContainer" containerID="f139e783431b1614f6a5c516d619f38ae07eb5ecdb48ddff40cde90e6d7bb1e9"
Sep 11 13:37:50.079161 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:50.079138    2554 scope.go:117] "RemoveContainer" containerID="f139e783431b1614f6a5c516d619f38ae07eb5ecdb48ddff40cde90e6d7bb1e9"
Sep 11 13:37:50.079544 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:37:50.079517    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f139e783431b1614f6a5c516d619f38ae07eb5ecdb48ddff40cde90e6d7bb1e9\": container with ID starting with f139e783431b1614f6a5c516d619f38ae07eb5ecdb48ddff40cde90e6d7bb1e9 not found: ID does not exist" containerID="f139e783431b1614f6a5c516d619f38ae07eb5ecdb48ddff40cde90e6d7bb1e9"
Sep 11 13:37:50.079634 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:50.079550    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f139e783431b1614f6a5c516d619f38ae07eb5ecdb48ddff40cde90e6d7bb1e9"} err="failed to get container status \"f139e783431b1614f6a5c516d619f38ae07eb5ecdb48ddff40cde90e6d7bb1e9\": rpc error: code = NotFound desc = could not find container \"f139e783431b1614f6a5c516d619f38ae07eb5ecdb48ddff40cde90e6d7bb1e9\": container with ID starting with f139e783431b1614f6a5c516d619f38ae07eb5ecdb48ddff40cde90e6d7bb1e9 not found: ID does not exist"
Sep 11 13:37:50.092594 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:50.092564    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"]
Sep 11 13:37:50.096567 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:50.096541    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6ffb6a-8c7c8dbf4-mlk9x"]
Sep 11 13:37:50.144755 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:50.144675    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:50.971395 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:50.971345    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:51.154187 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:51.154156    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30e0f013-8b18-42f9-94fe-6cdf5cc7b71b" path="/var/lib/kubelet/pods/30e0f013-8b18-42f9-94fe-6cdf5cc7b71b/volumes"
Sep 11 13:37:51.154561 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:51.154487    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de60e156-29f1-41cb-b76d-16d5c733875b" path="/var/lib/kubelet/pods/de60e156-29f1-41cb-b76d-16d5c733875b/volumes"
Sep 11 13:37:56.528274 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:56.528240    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:56.635517 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:56.635482    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:56.947580 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:56.947517    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:58.009535 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:58.009501    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:58.791541 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:37:58.791498    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:06.434556 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:06.434518    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:06.663832 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:06.663798    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:08.186325 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.186291    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"]
Sep 11 13:38:08.186738 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.186541    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4" podUID="cd9a8898-bbe1-4147-bbb9-25c94b80c6c9" containerName="maas-api" containerID="cri-o://a54ed36515a6036adc8c8c96b8023a1107ac24857c3d50943dfd3d23112fda66" gracePeriod=30
Sep 11 13:38:08.289490 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.289456    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:08.442421 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.442351    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:38:08.514769 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.514739    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-openshift-service-ca\") pod \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") "
Sep 11 13:38:08.514769 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.514775    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6jlz\" (UniqueName: \"kubernetes.io/projected/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-kube-api-access-w6jlz\") pod \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") "
Sep 11 13:38:08.514975 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.514826    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-metrics-certs\") pod \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") "
Sep 11 13:38:08.514975 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.514857    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-maas-api-tls\") pod \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\" (UID: \"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9\") "
Sep 11 13:38:08.515229 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.515201    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cd9a8898-bbe1-4147-bbb9-25c94b80c6c9" (UID: "cd9a8898-bbe1-4147-bbb9-25c94b80c6c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:38:08.517099 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.517064    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cd9a8898-bbe1-4147-bbb9-25c94b80c6c9" (UID: "cd9a8898-bbe1-4147-bbb9-25c94b80c6c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:38:08.517099 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.517084    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cd9a8898-bbe1-4147-bbb9-25c94b80c6c9" (UID: "cd9a8898-bbe1-4147-bbb9-25c94b80c6c9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:38:08.517237 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.517137    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-kube-api-access-w6jlz" (OuterVolumeSpecName: "kube-api-access-w6jlz") pod "cd9a8898-bbe1-4147-bbb9-25c94b80c6c9" (UID: "cd9a8898-bbe1-4147-bbb9-25c94b80c6c9"). InnerVolumeSpecName "kube-api-access-w6jlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:08.588319 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.588283    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:08.615628 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.615596    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:38:08.615628 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.615620    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:38:08.615628 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.615631    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:38:08.615865 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.615640    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6jlz\" (UniqueName: \"kubernetes.io/projected/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9-kube-api-access-w6jlz\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:38:08.722642 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.722560    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:08.850300 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.850261    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:08.951005 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:08.950963    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:09.173241 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:09.173207    2554 generic.go:356] "Generic (PLEG): container finished" podID="cd9a8898-bbe1-4147-bbb9-25c94b80c6c9" containerID="a54ed36515a6036adc8c8c96b8023a1107ac24857c3d50943dfd3d23112fda66" exitCode=0
Sep 11 13:38:09.173453 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:09.173268    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"
Sep 11 13:38:09.173453 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:09.173289    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4" event={"ID":"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9","Type":"ContainerDied","Data":"a54ed36515a6036adc8c8c96b8023a1107ac24857c3d50943dfd3d23112fda66"}
Sep 11 13:38:09.173453 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:09.173384    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4" event={"ID":"cd9a8898-bbe1-4147-bbb9-25c94b80c6c9","Type":"ContainerDied","Data":"483ed9990049d6cd95a1972cb06147c2aeee1f673b93d511150e2d16626d1ca8"}
Sep 11 13:38:09.173453 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:09.173408    2554 scope.go:117] "RemoveContainer" containerID="a54ed36515a6036adc8c8c96b8023a1107ac24857c3d50943dfd3d23112fda66"
Sep 11 13:38:09.182324 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:09.182301    2554 scope.go:117] "RemoveContainer" containerID="a54ed36515a6036adc8c8c96b8023a1107ac24857c3d50943dfd3d23112fda66"
Sep 11 13:38:09.182744 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:38:09.182694    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a54ed36515a6036adc8c8c96b8023a1107ac24857c3d50943dfd3d23112fda66\": container with ID starting with a54ed36515a6036adc8c8c96b8023a1107ac24857c3d50943dfd3d23112fda66 not found: ID does not exist" containerID="a54ed36515a6036adc8c8c96b8023a1107ac24857c3d50943dfd3d23112fda66"
Sep 11 13:38:09.182835 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:09.182733    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a54ed36515a6036adc8c8c96b8023a1107ac24857c3d50943dfd3d23112fda66"} err="failed to get container status \"a54ed36515a6036adc8c8c96b8023a1107ac24857c3d50943dfd3d23112fda66\": rpc error: code = NotFound desc = could not find container \"a54ed36515a6036adc8c8c96b8023a1107ac24857c3d50943dfd3d23112fda66\": container with ID starting with a54ed36515a6036adc8c8c96b8023a1107ac24857c3d50943dfd3d23112fda66 not found: ID does not exist"
Sep 11 13:38:09.210300 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:09.210262    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"]
Sep 11 13:38:09.217777 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:09.217741    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-12f386-685995b896-hgbw4"]
Sep 11 13:38:09.260544 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:09.260508    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:09.369011 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:09.368954    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:11.153441 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:11.153409    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd9a8898-bbe1-4147-bbb9-25c94b80c6c9" path="/var/lib/kubelet/pods/cd9a8898-bbe1-4147-bbb9-25c94b80c6c9/volumes"
Sep 11 13:38:17.118552 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:17.118512    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:17.430810 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:17.430735    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:17.728705 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:17.728611    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:37.388819 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:37.388788    2554 scope.go:117] "RemoveContainer" containerID="28acab3e64a4eb8f77fdcc6dd2baf15e0f9ee8d9bd5910b34c27dc4a3c1314de"
Sep 11 13:38:39.174741 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:38:39.174704    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:37.259077 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:39:37.259043    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:39:37.262742 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:39:37.262716    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:39:38.668404 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:39:38.668341    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:45.772450 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:39:45.772410    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:47.260557 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:39:47.260519    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:53.205530 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:41:53.205495    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:41.770812 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:42:41.770775    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:49.223973 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:42:49.223924    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:51.402359 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:42:51.402322    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:08.384489 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:08.384452    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:10.025680 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:10.025614    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:29.204838 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.204801    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"]
Sep 11 13:43:29.205256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.205151    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30e0f013-8b18-42f9-94fe-6cdf5cc7b71b" containerName="maas-api"
Sep 11 13:43:29.205256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.205162    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="30e0f013-8b18-42f9-94fe-6cdf5cc7b71b" containerName="maas-api"
Sep 11 13:43:29.205256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.205173    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2d24189-1e24-4de8-a86e-6fdff2dcf67e" containerName="maas-api"
Sep 11 13:43:29.205256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.205178    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2d24189-1e24-4de8-a86e-6fdff2dcf67e" containerName="maas-api"
Sep 11 13:43:29.205256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.205191    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd9a8898-bbe1-4147-bbb9-25c94b80c6c9" containerName="maas-api"
Sep 11 13:43:29.205256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.205196    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd9a8898-bbe1-4147-bbb9-25c94b80c6c9" containerName="maas-api"
Sep 11 13:43:29.205256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.205203    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aac145f9-c8c8-496e-93e7-d3a03cce9049" containerName="maas-api"
Sep 11 13:43:29.205256 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.205208    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="aac145f9-c8c8-496e-93e7-d3a03cce9049" containerName="maas-api"
Sep 11 13:43:29.205541 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.205263    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="30e0f013-8b18-42f9-94fe-6cdf5cc7b71b" containerName="maas-api"
Sep 11 13:43:29.205541 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.205271    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd9a8898-bbe1-4147-bbb9-25c94b80c6c9" containerName="maas-api"
Sep 11 13:43:29.205541 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.205279    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2d24189-1e24-4de8-a86e-6fdff2dcf67e" containerName="maas-api"
Sep 11 13:43:29.205541 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.205285    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="aac145f9-c8c8-496e-93e7-d3a03cce9049" containerName="maas-api"
Sep 11 13:43:29.208162 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.208142    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.211593 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.211567    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 13:43:29.211698 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.211589    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 13:43:29.211698 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.211592    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 13:43:29.211785 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.211707    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zxpw2\""
Sep 11 13:43:29.211879 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.211864    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:43:29.231462 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.231426    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"]
Sep 11 13:43:29.295072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.295033    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877157e0-7782-4d60-8152-a5733e050a27-metrics-certs\") pod \"maas-api-549f4476fd-2hcnh\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.295072 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.295072    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn2hd\" (UniqueName: \"kubernetes.io/projected/877157e0-7782-4d60-8152-a5733e050a27-kube-api-access-jn2hd\") pod \"maas-api-549f4476fd-2hcnh\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.295293 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.295104    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/877157e0-7782-4d60-8152-a5733e050a27-maas-api-tls\") pod \"maas-api-549f4476fd-2hcnh\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.295293 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.295216    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/877157e0-7782-4d60-8152-a5733e050a27-openshift-service-ca\") pod \"maas-api-549f4476fd-2hcnh\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.395856 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.395813    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/877157e0-7782-4d60-8152-a5733e050a27-openshift-service-ca\") pod \"maas-api-549f4476fd-2hcnh\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.395856 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.395861    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877157e0-7782-4d60-8152-a5733e050a27-metrics-certs\") pod \"maas-api-549f4476fd-2hcnh\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.396129 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.395880    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn2hd\" (UniqueName: \"kubernetes.io/projected/877157e0-7782-4d60-8152-a5733e050a27-kube-api-access-jn2hd\") pod \"maas-api-549f4476fd-2hcnh\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.396129 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.395910    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/877157e0-7782-4d60-8152-a5733e050a27-maas-api-tls\") pod \"maas-api-549f4476fd-2hcnh\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.396565 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.396541    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/877157e0-7782-4d60-8152-a5733e050a27-openshift-service-ca\") pod \"maas-api-549f4476fd-2hcnh\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.398563 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.398540    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877157e0-7782-4d60-8152-a5733e050a27-metrics-certs\") pod \"maas-api-549f4476fd-2hcnh\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.398563 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.398556    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/877157e0-7782-4d60-8152-a5733e050a27-maas-api-tls\") pod \"maas-api-549f4476fd-2hcnh\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.405580 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.405551    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn2hd\" (UniqueName: \"kubernetes.io/projected/877157e0-7782-4d60-8152-a5733e050a27-kube-api-access-jn2hd\") pod \"maas-api-549f4476fd-2hcnh\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.518795 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.518704    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:29.798434 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.798408    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"]
Sep 11 13:43:29.801308 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:43:29.801278    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877157e0_7782_4d60_8152_a5733e050a27.slice/crio-d2305e10e68deb14caf93fbf88a80f2cbaa5c33bd702ce7f66c66ae96a2e27ca WatchSource:0}: Error finding container d2305e10e68deb14caf93fbf88a80f2cbaa5c33bd702ce7f66c66ae96a2e27ca: Status 404 returned error can't find the container with id d2305e10e68deb14caf93fbf88a80f2cbaa5c33bd702ce7f66c66ae96a2e27ca
Sep 11 13:43:29.803130 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:29.803112    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:43:30.741448 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:30.741409    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh" event={"ID":"877157e0-7782-4d60-8152-a5733e050a27","Type":"ContainerStarted","Data":"93db68bdbcb048a384744ac65da1371cbb5cd1ad2223de8fb5490fe26ef284a8"}
Sep 11 13:43:30.741899 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:30.741466    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh" event={"ID":"877157e0-7782-4d60-8152-a5733e050a27","Type":"ContainerStarted","Data":"d2305e10e68deb14caf93fbf88a80f2cbaa5c33bd702ce7f66c66ae96a2e27ca"}
Sep 11 13:43:30.741899 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:30.741513    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:30.762330 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:30.762265    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh" podStartSLOduration=1.31856704 podStartE2EDuration="1.762242482s" totalImagesPullingTime="443.675442ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:43:30.761554802 +0000 UTC m=+2334.079011844" watchObservedRunningTime="2026-09-11 13:43:30.762242482 +0000 UTC m=+2334.079699522"
Sep 11 13:43:36.752644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:36.752595    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:36.807596 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:36.807563    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"]
Sep 11 13:43:36.807829 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:36.807801    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh" podUID="877157e0-7782-4d60-8152-a5733e050a27" containerName="maas-api" containerID="cri-o://93db68bdbcb048a384744ac65da1371cbb5cd1ad2223de8fb5490fe26ef284a8" gracePeriod=30
Sep 11 13:43:36.854305 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:36.854268    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"]
Sep 11 13:43:36.857736 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:36.857711    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:36.879099 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:36.879067    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"]
Sep 11 13:43:36.958437 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:36.958120    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llthp\" (UniqueName: \"kubernetes.io/projected/8642057b-9dc7-464a-b3a9-3a73fc4ece55-kube-api-access-llthp\") pod \"maas-api-6876b74b58-lfhl2\" (UID: \"8642057b-9dc7-464a-b3a9-3a73fc4ece55\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:36.958437 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:36.958183    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8642057b-9dc7-464a-b3a9-3a73fc4ece55-openshift-service-ca\") pod \"maas-api-6876b74b58-lfhl2\" (UID: \"8642057b-9dc7-464a-b3a9-3a73fc4ece55\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:36.958437 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:36.958216    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8642057b-9dc7-464a-b3a9-3a73fc4ece55-maas-api-tls\") pod \"maas-api-6876b74b58-lfhl2\" (UID: \"8642057b-9dc7-464a-b3a9-3a73fc4ece55\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:36.958437 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:36.958244    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8642057b-9dc7-464a-b3a9-3a73fc4ece55-metrics-certs\") pod \"maas-api-6876b74b58-lfhl2\" (UID: \"8642057b-9dc7-464a-b3a9-3a73fc4ece55\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:37.059584 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.059546    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8642057b-9dc7-464a-b3a9-3a73fc4ece55-openshift-service-ca\") pod \"maas-api-6876b74b58-lfhl2\" (UID: \"8642057b-9dc7-464a-b3a9-3a73fc4ece55\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:37.059742 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.059589    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8642057b-9dc7-464a-b3a9-3a73fc4ece55-maas-api-tls\") pod \"maas-api-6876b74b58-lfhl2\" (UID: \"8642057b-9dc7-464a-b3a9-3a73fc4ece55\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:37.059742 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.059607    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8642057b-9dc7-464a-b3a9-3a73fc4ece55-metrics-certs\") pod \"maas-api-6876b74b58-lfhl2\" (UID: \"8642057b-9dc7-464a-b3a9-3a73fc4ece55\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:37.059742 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.059707    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llthp\" (UniqueName: \"kubernetes.io/projected/8642057b-9dc7-464a-b3a9-3a73fc4ece55-kube-api-access-llthp\") pod \"maas-api-6876b74b58-lfhl2\" (UID: \"8642057b-9dc7-464a-b3a9-3a73fc4ece55\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:37.060232 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.060210    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8642057b-9dc7-464a-b3a9-3a73fc4ece55-openshift-service-ca\") pod \"maas-api-6876b74b58-lfhl2\" (UID: \"8642057b-9dc7-464a-b3a9-3a73fc4ece55\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:37.062168 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.062135    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8642057b-9dc7-464a-b3a9-3a73fc4ece55-maas-api-tls\") pod \"maas-api-6876b74b58-lfhl2\" (UID: \"8642057b-9dc7-464a-b3a9-3a73fc4ece55\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:37.062334 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.062317    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8642057b-9dc7-464a-b3a9-3a73fc4ece55-metrics-certs\") pod \"maas-api-6876b74b58-lfhl2\" (UID: \"8642057b-9dc7-464a-b3a9-3a73fc4ece55\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:37.069108 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.069082    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llthp\" (UniqueName: \"kubernetes.io/projected/8642057b-9dc7-464a-b3a9-3a73fc4ece55-kube-api-access-llthp\") pod \"maas-api-6876b74b58-lfhl2\" (UID: \"8642057b-9dc7-464a-b3a9-3a73fc4ece55\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:37.083580 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.083553    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:37.160726 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.160694    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/877157e0-7782-4d60-8152-a5733e050a27-openshift-service-ca\") pod \"877157e0-7782-4d60-8152-a5733e050a27\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") "
Sep 11 13:43:37.160897 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.160764    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/877157e0-7782-4d60-8152-a5733e050a27-maas-api-tls\") pod \"877157e0-7782-4d60-8152-a5733e050a27\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") "
Sep 11 13:43:37.160897 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.160808    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn2hd\" (UniqueName: \"kubernetes.io/projected/877157e0-7782-4d60-8152-a5733e050a27-kube-api-access-jn2hd\") pod \"877157e0-7782-4d60-8152-a5733e050a27\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") "
Sep 11 13:43:37.160897 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.160862    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877157e0-7782-4d60-8152-a5733e050a27-metrics-certs\") pod \"877157e0-7782-4d60-8152-a5733e050a27\" (UID: \"877157e0-7782-4d60-8152-a5733e050a27\") "
Sep 11 13:43:37.161139 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.161114    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/877157e0-7782-4d60-8152-a5733e050a27-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "877157e0-7782-4d60-8152-a5733e050a27" (UID: "877157e0-7782-4d60-8152-a5733e050a27"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:43:37.162782 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.162759    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/877157e0-7782-4d60-8152-a5733e050a27-kube-api-access-jn2hd" (OuterVolumeSpecName: "kube-api-access-jn2hd") pod "877157e0-7782-4d60-8152-a5733e050a27" (UID: "877157e0-7782-4d60-8152-a5733e050a27"). InnerVolumeSpecName "kube-api-access-jn2hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:43:37.162842 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.162792    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/877157e0-7782-4d60-8152-a5733e050a27-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "877157e0-7782-4d60-8152-a5733e050a27" (UID: "877157e0-7782-4d60-8152-a5733e050a27"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:43:37.162842 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.162810    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/877157e0-7782-4d60-8152-a5733e050a27-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "877157e0-7782-4d60-8152-a5733e050a27" (UID: "877157e0-7782-4d60-8152-a5733e050a27"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:43:37.177011 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.176985    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:37.261820 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.261792    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877157e0-7782-4d60-8152-a5733e050a27-metrics-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:43:37.261820 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.261819    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/877157e0-7782-4d60-8152-a5733e050a27-openshift-service-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:43:37.261993 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.261829    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/877157e0-7782-4d60-8152-a5733e050a27-maas-api-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:43:37.261993 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.261839    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn2hd\" (UniqueName: \"kubernetes.io/projected/877157e0-7782-4d60-8152-a5733e050a27-kube-api-access-jn2hd\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Sep 11 13:43:37.472477 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.472356    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"]
Sep 11 13:43:37.475012 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:43:37.474985    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8642057b_9dc7_464a_b3a9_3a73fc4ece55.slice/crio-b4d4dd4c9fa1fee1552d8c2a3e718da206ba4a39fc66f77365fed4f7dd412d2e WatchSource:0}: Error finding container b4d4dd4c9fa1fee1552d8c2a3e718da206ba4a39fc66f77365fed4f7dd412d2e: Status 404 returned error can't find the container with id b4d4dd4c9fa1fee1552d8c2a3e718da206ba4a39fc66f77365fed4f7dd412d2e
Sep 11 13:43:37.779956 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.779860    2554 generic.go:356] "Generic (PLEG): container finished" podID="877157e0-7782-4d60-8152-a5733e050a27" containerID="93db68bdbcb048a384744ac65da1371cbb5cd1ad2223de8fb5490fe26ef284a8" exitCode=0
Sep 11 13:43:37.779956 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.779912    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"
Sep 11 13:43:37.780478 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.779903    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh" event={"ID":"877157e0-7782-4d60-8152-a5733e050a27","Type":"ContainerDied","Data":"93db68bdbcb048a384744ac65da1371cbb5cd1ad2223de8fb5490fe26ef284a8"}
Sep 11 13:43:37.780478 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.780084    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh" event={"ID":"877157e0-7782-4d60-8152-a5733e050a27","Type":"ContainerDied","Data":"d2305e10e68deb14caf93fbf88a80f2cbaa5c33bd702ce7f66c66ae96a2e27ca"}
Sep 11 13:43:37.780478 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.780120    2554 scope.go:117] "RemoveContainer" containerID="93db68bdbcb048a384744ac65da1371cbb5cd1ad2223de8fb5490fe26ef284a8"
Sep 11 13:43:37.781353 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.781329    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2" event={"ID":"8642057b-9dc7-464a-b3a9-3a73fc4ece55","Type":"ContainerStarted","Data":"b4d4dd4c9fa1fee1552d8c2a3e718da206ba4a39fc66f77365fed4f7dd412d2e"}
Sep 11 13:43:37.789032 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.789014    2554 scope.go:117] "RemoveContainer" containerID="93db68bdbcb048a384744ac65da1371cbb5cd1ad2223de8fb5490fe26ef284a8"
Sep 11 13:43:37.789296 ip-10-0-143-79 kubenswrapper[2554]: E0911 13:43:37.789277    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93db68bdbcb048a384744ac65da1371cbb5cd1ad2223de8fb5490fe26ef284a8\": container with ID starting with 93db68bdbcb048a384744ac65da1371cbb5cd1ad2223de8fb5490fe26ef284a8 not found: ID does not exist" containerID="93db68bdbcb048a384744ac65da1371cbb5cd1ad2223de8fb5490fe26ef284a8"
Sep 11 13:43:37.789344 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.789304    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93db68bdbcb048a384744ac65da1371cbb5cd1ad2223de8fb5490fe26ef284a8"} err="failed to get container status \"93db68bdbcb048a384744ac65da1371cbb5cd1ad2223de8fb5490fe26ef284a8\": rpc error: code = NotFound desc = could not find container \"93db68bdbcb048a384744ac65da1371cbb5cd1ad2223de8fb5490fe26ef284a8\": container with ID starting with 93db68bdbcb048a384744ac65da1371cbb5cd1ad2223de8fb5490fe26ef284a8 not found: ID does not exist"
Sep 11 13:43:37.803348 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.803320    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"]
Sep 11 13:43:37.809727 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:37.809707    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-549f4476fd-2hcnh"]
Sep 11 13:43:38.787835 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:38.787792    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2" event={"ID":"8642057b-9dc7-464a-b3a9-3a73fc4ece55","Type":"ContainerStarted","Data":"198c7c05b15d5eb6a4b3fcca5c9b38e0d9109e29061f3c72d54622322ac6d86d"}
Sep 11 13:43:38.788554 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:38.787898    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:38.811809 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:38.811760    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2" podStartSLOduration=2.269660892 podStartE2EDuration="2.811743958s" totalImagesPullingTime="542.083066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:43:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:43:38.810163495 +0000 UTC m=+2342.127620537" watchObservedRunningTime="2026-09-11 13:43:38.811743958 +0000 UTC m=+2342.129200998"
Sep 11 13:43:38.820417 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:38.820387    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:39.153757 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:39.153723    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="877157e0-7782-4d60-8152-a5733e050a27" path="/var/lib/kubelet/pods/877157e0-7782-4d60-8152-a5733e050a27/volumes"
Sep 11 13:43:44.799190 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:44.799157    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6876b74b58-lfhl2"
Sep 11 13:43:48.940935 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:48.940900    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:49.008517 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:43:49.008483    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:04.969596 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:04.969559    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-574z2_1fd996fe-1e9a-4a82-a37a-d1161f54177b/server/0.log"
Sep 11 13:44:05.214004 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:05.213972    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b86887d-fjpcd_aa961889-771b-4937-a1fb-d59980ee0b1e/copy-manifests/0.log"
Sep 11 13:44:05.220588 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:05.220527    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b86887d-fjpcd_aa961889-771b-4937-a1fb-d59980ee0b1e/manager/0.log"
Sep 11 13:44:05.956611 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:05.956582    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6876b74b58-lfhl2_8642057b-9dc7-464a-b3a9-3a73fc4ece55/maas-api/0.log"
Sep 11 13:44:08.185961 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:08.185927    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4gmk5_0b62a2b8-0ec4-4a9d-8816-8350c8a49367/discovery/0.log"
Sep 11 13:44:08.302754 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:08.302723    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74db4b4cbd-b5qwr_4e27e323-a30b-4bc3-8b06-437d4ada1fb9/kube-auth-proxy/0.log"
Sep 11 13:44:08.901864 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:08.901832    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b946bb9b8-kxhmt_a282b450-2ac0-4aed-98ac-7be36ff01407/router/0.log"
Sep 11 13:44:15.534403 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:15.534354    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:15.659747 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:15.659715    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:18.719320 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:18.719280    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p5hww_c6942647-056e-4b32-a2b4-ea2e90f7be76/global-pull-secret-syncer/0.log"
Sep 11 13:44:18.770548 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:18.770514    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8hpxh_00d20a97-d950-479a-9e5d-0ebb126e3e9c/konnectivity-agent/0.log"
Sep 11 13:44:18.938470 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:18.938443    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-79.ec2.internal_9c13e4afb5118b6c262250604375f150/haproxy/0.log"
Sep 11 13:44:25.916916 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:25.916867    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-4rqbt_6e4b7f10-74fd-40fd-ba55-61080037f297/cluster-monitoring-operator/0.log"
Sep 11 13:44:25.961315 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:25.961281    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-qmw72_453275be-2adc-4e7b-aed8-cf4e83a6165b/kube-state-metrics/0.log"
Sep 11 13:44:26.000179 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:26.000145    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-qmw72_453275be-2adc-4e7b-aed8-cf4e83a6165b/kube-rbac-proxy-main/0.log"
Sep 11 13:44:26.045316 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:26.045282    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-qmw72_453275be-2adc-4e7b-aed8-cf4e83a6165b/kube-rbac-proxy-self/0.log"
Sep 11 13:44:26.212606 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:26.212530    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-sp56g_67d490a6-167c-4a9c-9d94-e5ca6eb44cd0/monitoring-plugin/0.log"
Sep 11 13:44:26.288344 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:26.288313    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-77bk5_13816e24-f2f6-49d4-8f05-d3f5c23adc94/node-exporter/0.log"
Sep 11 13:44:26.391399 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:26.391351    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-77bk5_13816e24-f2f6-49d4-8f05-d3f5c23adc94/kube-rbac-proxy/0.log"
Sep 11 13:44:26.481135 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:26.481055    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-77bk5_13816e24-f2f6-49d4-8f05-d3f5c23adc94/init-textfile/0.log"
Sep 11 13:44:27.015487 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:27.015457    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-xnhqn_7e6cd0e1-fff0-44de-84dc-9fb32fb6c962/kube-rbac-proxy-main/0.log"
Sep 11 13:44:27.054105 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:27.054079    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-xnhqn_7e6cd0e1-fff0-44de-84dc-9fb32fb6c962/kube-rbac-proxy-self/0.log"
Sep 11 13:44:27.087437 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:27.087405    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-xnhqn_7e6cd0e1-fff0-44de-84dc-9fb32fb6c962/openshift-state-metrics/0.log"
Sep 11 13:44:27.580993 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:27.580958    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-9sbhl_3165da23-31f2-4da8-80d6-90d815044bfa/prometheus-operator/0.log"
Sep 11 13:44:27.704303 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:27.704270    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-9sbhl_3165da23-31f2-4da8-80d6-90d815044bfa/kube-rbac-proxy/0.log"
Sep 11 13:44:28.206178 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.206136    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"]
Sep 11 13:44:28.206774 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.206751    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="877157e0-7782-4d60-8152-a5733e050a27" containerName="maas-api"
Sep 11 13:44:28.206774 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.206773    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="877157e0-7782-4d60-8152-a5733e050a27" containerName="maas-api"
Sep 11 13:44:28.206894 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.206848    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="877157e0-7782-4d60-8152-a5733e050a27" containerName="maas-api"
Sep 11 13:44:28.209710 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.209692    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.214089 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.214067    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5f924\"/\"kube-root-ca.crt\""
Sep 11 13:44:28.214990 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.214964    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5f924\"/\"default-dockercfg-cxjdm\""
Sep 11 13:44:28.215138 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.214980    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5f924\"/\"openshift-service-ca.crt\""
Sep 11 13:44:28.225651 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.225625    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"]
Sep 11 13:44:28.301150 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.301111    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s62rt\" (UniqueName: \"kubernetes.io/projected/dbf30969-d52c-4f92-a547-7701c2c8d00d-kube-api-access-s62rt\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.301339 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.301160    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dbf30969-d52c-4f92-a547-7701c2c8d00d-proc\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.301339 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.301212    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dbf30969-d52c-4f92-a547-7701c2c8d00d-podres\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.301339 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.301236    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbf30969-d52c-4f92-a547-7701c2c8d00d-lib-modules\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.301339 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.301267    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbf30969-d52c-4f92-a547-7701c2c8d00d-sys\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.402092 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.402054    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s62rt\" (UniqueName: \"kubernetes.io/projected/dbf30969-d52c-4f92-a547-7701c2c8d00d-kube-api-access-s62rt\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.402281 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.402107    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dbf30969-d52c-4f92-a547-7701c2c8d00d-proc\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.402281 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.402233    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dbf30969-d52c-4f92-a547-7701c2c8d00d-podres\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.402433 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.402278    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dbf30969-d52c-4f92-a547-7701c2c8d00d-proc\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.402433 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.402287    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbf30969-d52c-4f92-a547-7701c2c8d00d-lib-modules\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.402433 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.402329    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbf30969-d52c-4f92-a547-7701c2c8d00d-sys\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.402433 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.402397    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dbf30969-d52c-4f92-a547-7701c2c8d00d-podres\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.402433 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.402431    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbf30969-d52c-4f92-a547-7701c2c8d00d-sys\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.402644 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.402436    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbf30969-d52c-4f92-a547-7701c2c8d00d-lib-modules\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.412168 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.412142    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s62rt\" (UniqueName: \"kubernetes.io/projected/dbf30969-d52c-4f92-a547-7701c2c8d00d-kube-api-access-s62rt\") pod \"perf-node-gather-daemonset-pt9dl\" (UID: \"dbf30969-d52c-4f92-a547-7701c2c8d00d\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.519728 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.519639    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:28.816340 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:28.816314    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"]
Sep 11 13:44:28.819010 ip-10-0-143-79 kubenswrapper[2554]: W0911 13:44:28.818985    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddbf30969_d52c_4f92_a547_7701c2c8d00d.slice/crio-d9194085ce5e502aefa603ed1b593a366929eaed664fc18bab804fee7e182fcb WatchSource:0}: Error finding container d9194085ce5e502aefa603ed1b593a366929eaed664fc18bab804fee7e182fcb: Status 404 returned error can't find the container with id d9194085ce5e502aefa603ed1b593a366929eaed664fc18bab804fee7e182fcb
Sep 11 13:44:29.035694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:29.035658    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl" event={"ID":"dbf30969-d52c-4f92-a547-7701c2c8d00d","Type":"ContainerStarted","Data":"da1ab6c92b890adc3cf58129fefe77f5af21c0c24e76b5648875f681346a86b7"}
Sep 11 13:44:29.035860 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:29.035709    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl" event={"ID":"dbf30969-d52c-4f92-a547-7701c2c8d00d","Type":"ContainerStarted","Data":"d9194085ce5e502aefa603ed1b593a366929eaed664fc18bab804fee7e182fcb"}
Sep 11 13:44:29.035860 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:29.035748    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:30.180972 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:30.180940    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:44:30.186344 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:30.186315    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/2.log"
Sep 11 13:44:31.316736 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:31.316707    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-jfl55_aa53386d-9f0f-4d4e-868f-520ca3f17cde/volume-data-source-validator/0.log"
Sep 11 13:44:32.435272 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:32.435197    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4hb2m_54af5720-e75b-4e36-94cc-4a72fb166eb4/dns/0.log"
Sep 11 13:44:32.465722 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:32.465694    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4hb2m_54af5720-e75b-4e36-94cc-4a72fb166eb4/kube-rbac-proxy/0.log"
Sep 11 13:44:32.653960 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:32.653928    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-twqp9_3b49a550-e226-4de1-8351-a9fd45b4f76f/dns-node-resolver/0.log"
Sep 11 13:44:33.362513 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:33.362480    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x89jp_1572be9a-6852-40f5-951c-b0a82cd9961f/node-ca/0.log"
Sep 11 13:44:34.961350 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:34.961311    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4gmk5_0b62a2b8-0ec4-4a9d-8816-8350c8a49367/discovery/0.log"
Sep 11 13:44:35.011358 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:35.011325    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74db4b4cbd-b5qwr_4e27e323-a30b-4bc3-8b06-437d4ada1fb9/kube-auth-proxy/0.log"
Sep 11 13:44:35.050835 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:35.050805    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl"
Sep 11 13:44:35.076225 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:35.076166    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-pt9dl" podStartSLOduration=7.076149463 podStartE2EDuration="7.076149463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:44:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:44:29.065776194 +0000 UTC m=+2392.383233234" watchObservedRunningTime="2026-09-11 13:44:35.076149463 +0000 UTC m=+2398.393606502"
Sep 11 13:44:35.416231 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:35.416196    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b946bb9b8-kxhmt_a282b450-2ac0-4aed-98ac-7be36ff01407/router/0.log"
Sep 11 13:44:36.882477 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:36.882450    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l7wdt_10e1239e-56fb-489b-9079-7b3c2e3619f9/serve-healthcheck-canary/0.log"
Sep 11 13:44:37.294451 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:37.294424    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:44:37.298893 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:37.298870    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n5c4g_a975d99e-85e6-4e33-b286-149c224c5d6e/console-operator/1.log"
Sep 11 13:44:37.800393 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:37.800297    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-p49pp_7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e/insights-operator/0.log"
Sep 11 13:44:37.801903 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:37.801882    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-p49pp_7cf01a79-1e79-45b8-8cd5-1ee9e4e8117e/insights-operator/1.log"
Sep 11 13:44:38.438850 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:38.438814    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mp69k_663c3e69-ad92-4794-b0c4-9b8c319b176c/kube-rbac-proxy/0.log"
Sep 11 13:44:38.505398 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:38.505359    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mp69k_663c3e69-ad92-4794-b0c4-9b8c319b176c/exporter/0.log"
Sep 11 13:44:38.596658 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:38.596624    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mp69k_663c3e69-ad92-4794-b0c4-9b8c319b176c/extractor/0.log"
Sep 11 13:44:38.774956 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:38.774926    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4gmk5_0b62a2b8-0ec4-4a9d-8816-8350c8a49367/discovery/0.log"
Sep 11 13:44:38.790981 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:38.790954    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74db4b4cbd-b5qwr_4e27e323-a30b-4bc3-8b06-437d4ada1fb9/kube-auth-proxy/0.log"
Sep 11 13:44:38.949164 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:38.949129    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b946bb9b8-kxhmt_a282b450-2ac0-4aed-98ac-7be36ff01407/router/0.log"
Sep 11 13:44:40.680670 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:40.680639    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4gmk5_0b62a2b8-0ec4-4a9d-8816-8350c8a49367/discovery/0.log"
Sep 11 13:44:40.703656 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:40.703631    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74db4b4cbd-b5qwr_4e27e323-a30b-4bc3-8b06-437d4ada1fb9/kube-auth-proxy/0.log"
Sep 11 13:44:40.877326 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:40.877298    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b946bb9b8-kxhmt_a282b450-2ac0-4aed-98ac-7be36ff01407/router/0.log"
Sep 11 13:44:41.747790 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:41.747738    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-574z2_1fd996fe-1e9a-4a82-a37a-d1161f54177b/server/0.log"
Sep 11 13:44:41.930908 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:41.930872    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b86887d-fjpcd_aa961889-771b-4937-a1fb-d59980ee0b1e/manager/0.log"
Sep 11 13:44:41.987694 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:41.987666    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b86887d-fjpcd_aa961889-771b-4937-a1fb-d59980ee0b1e/copy-manifests/0.log"
Sep 11 13:44:43.332333 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:43.332283    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5f8bd79dcc-nz8qv_8a6d1998-9e69-44db-9662-a8f8d4700f7f/manager/0.log"
Sep 11 13:44:43.427750 ip-10-0-143-79 kubenswrapper[2554]: I0911 13:44:43.427717    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-t6bxb_76b6a8a8-c009-4818-a7cb-8a2b77a82e57/openshift-lws-operator/0.log"