Jul 20 16:45:18.999421 ip-10-0-128-8 systemd[1]: Starting Kubernetes Kubelet...
Jul 20 16:45:19.331847 ip-10-0-128-8 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 16:45:19.331847 ip-10-0-128-8 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 20 16:45:19.331847 ip-10-0-128-8 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 16:45:19.331847 ip-10-0-128-8 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 20 16:45:19.331847 ip-10-0-128-8 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 16:45:19.333785 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.333709    2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 20 16:45:19.338115 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338093    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 16:45:19.338115 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338114    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 16:45:19.338115 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338120    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338125    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338129    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338133    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338137    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338141    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338165    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338170    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338174    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338178    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338182    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338185    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338196    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338201    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338205    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338208    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338212    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338216    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338220    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338224    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 16:45:19.338332 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338228    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338232    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338236    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338241    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338245    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338249    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338254    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338258    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338263    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338267    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338272    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338276    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338280    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338284    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338289    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338294    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338298    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338303    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338307    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338311    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 16:45:19.339171 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338316    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338321    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338325    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338329    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338333    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338338    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338342    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338346    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338351    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338354    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338361    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338367    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338371    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338376    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338380    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338384    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338389    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338393    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338401    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338407    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 16:45:19.339986 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338412    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338416    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338421    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338426    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338430    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338435    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338440    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338445    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338450    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338454    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338458    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338462    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338466    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338470    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338474    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338478    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338482    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338487    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338492    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 16:45:19.340811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338496    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338500    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338504    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338508    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338512    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338517    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338521    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338525    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338529    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338534    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338538    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338542    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338546    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338550    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338554    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338558    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338562    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.338567    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339728    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339738    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 16:45:19.341604 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339744    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339748    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339752    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339756    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339761    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339765    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339769    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339773    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339777    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339781    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339785    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339790    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339795    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339800    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339804    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339808    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339812    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339816    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339820    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339824    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 16:45:19.342498 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339828    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339832    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339836    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339840    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339845    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339849    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339854    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339857    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339862    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339866    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339870    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339875    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339880    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339884    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339888    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339891    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339895    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339899    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339903    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 16:45:19.343275 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339907    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339911    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339914    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339918    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339922    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339926    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339932    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339936    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339939    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339944    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339948    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339952    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339956    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339961    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339967    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339971    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339975    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339979    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339983    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339988    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 16:45:19.343835 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339991    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339996    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.339999    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340003    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340007    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340011    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340015    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340019    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340023    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340027    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340031    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340035    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340039    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340043    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340047    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340051    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340055    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340058    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340062    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 16:45:19.344458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340067    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340072    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340076    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340080    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340084    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340088    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340092    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340096    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340103    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340110    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340115    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340120    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340125    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340129    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340133    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340140    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340165    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340171    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 16:45:19.344912 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340176    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340759    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340767    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340773    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340778    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340783    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340787    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340792    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340796    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340800    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340804    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340808    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340812    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340815    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340820    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340826    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340831    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340836    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340841    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340846    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 16:45:19.345466 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340851    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340856    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340860    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340865    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340869    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340873    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340878    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340882    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340886    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340890    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340894    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340898    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340902    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340908    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340913    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340917    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340920    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340924    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340928    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 16:45:19.345990 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340932    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340936    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340940    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340944    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340948    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340972    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340978    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340984    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340990    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340994    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.340999    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341003    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341007    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341011    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341016    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341022    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341028    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341034    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341038    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341043    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 16:45:19.346487 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341048    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341052    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341058    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341062    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341067    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341072    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341076    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341080    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341085    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341089    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341093    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341097    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341101    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341105    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341108    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341112    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341116    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341120    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341124    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341128    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 16:45:19.347008 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341135    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341162    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341168    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341172    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341178    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341181    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341185    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341189    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341193    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341197    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341202    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341206    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341209    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341213    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341217    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341221    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341226    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341230    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341233    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 16:45:19.347512 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341238    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.341242    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341332    2568 flags.go:64] FLAG: --address="0.0.0.0"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341342    2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341351    2568 flags.go:64] FLAG: --anonymous-auth="true"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341358    2568 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341364    2568 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341368    2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341375    2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341381    2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341386    2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341391    2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341396    2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341406    2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341411    2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341416    2568 flags.go:64] FLAG: --cgroup-root=""
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341421    2568 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341426    2568 flags.go:64] FLAG: --client-ca-file=""
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341430    2568 flags.go:64] FLAG: --cloud-provider="external"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341434    2568 flags.go:64] FLAG: --cluster-dns="[]"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341439    2568 flags.go:64] FLAG: --cluster-domain=""
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341444    2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341448    2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341453    2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 20 16:45:19.347952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341458    2568 flags.go:64] FLAG: --container-log-max-files="5"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341464    2568 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341468    2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341474    2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341478    2568 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341483    2568 flags.go:64] FLAG: --contention-profiling="false"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341487    2568 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341491    2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341496    2568 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341501    2568 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341507    2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341512    2568 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341516    2568 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341521    2568 flags.go:64] FLAG: --enable-load-reader="false"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341525    2568 flags.go:64] FLAG: --enable-server="true"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341529    2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341535    2568 flags.go:64] FLAG: --event-burst="100"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341540    2568 flags.go:64] FLAG: --event-qps="50"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341544    2568 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341549    2568 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341553    2568 flags.go:64] FLAG: --eviction-hard=""
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341562    2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341566    2568 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341570    2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341575    2568 flags.go:64] FLAG: --eviction-soft=""
Jul 20 16:45:19.348652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341580    2568 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341584    2568 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341589    2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341593    2568 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341597    2568 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341601    2568 flags.go:64] FLAG: --fail-swap-on="true"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341605    2568 flags.go:64] FLAG: --feature-gates=""
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341610    2568 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341615    2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341620    2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341624    2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341629    2568 flags.go:64] FLAG: --healthz-port="10248"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341633    2568 flags.go:64] FLAG: --help="false"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341638    2568 flags.go:64] FLAG: --hostname-override="ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341642    2568 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341647    2568 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341651    2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341656    2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341662    2568 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341666    2568 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341670    2568 flags.go:64] FLAG: --image-service-endpoint=""
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341674    2568 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341679    2568 flags.go:64] FLAG: --kube-api-burst="100"
Jul 20 16:45:19.349587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341683    2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341688    2568 flags.go:64] FLAG: --kube-api-qps="50"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341692    2568 flags.go:64] FLAG: --kube-reserved=""
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341697    2568 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341702    2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341710    2568 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341714    2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341719    2568 flags.go:64] FLAG: --lock-file=""
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341723    2568 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341728    2568 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341733    2568 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341741    2568 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341746    2568 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341750    2568 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341754    2568 flags.go:64] FLAG: --logging-format="text"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341758    2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341764    2568 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341769    2568 flags.go:64] FLAG: --manifest-url=""
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341773    2568 flags.go:64] FLAG: --manifest-url-header=""
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341780    2568 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341784    2568 flags.go:64] FLAG: --max-open-files="1000000"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341790    2568 flags.go:64] FLAG: --max-pods="110"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341795    2568 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341799    2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341804    2568 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 20 16:45:19.350264 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341809    2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341814    2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341818    2568 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341823    2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341837    2568 flags.go:64] FLAG: --node-status-max-images="50"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341841    2568 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341846    2568 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341850    2568 flags.go:64] FLAG: --pod-cidr=""
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341855    2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341863    2568 flags.go:64] FLAG: --pod-manifest-path=""
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341867    2568 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341872    2568 flags.go:64] FLAG: --pods-per-core="0"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341887    2568 flags.go:64] FLAG: --port="10250"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341894    2568 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341899    2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07ecf12f55db15638"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341904    2568 flags.go:64] FLAG: --qos-reserved=""
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341909    2568 flags.go:64] FLAG: --read-only-port="10255"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341932    2568 flags.go:64] FLAG: --register-node="true"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341937    2568 flags.go:64] FLAG: --register-with-taints=""
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341943    2568 flags.go:64] FLAG: --registry-burst="10"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341947    2568 flags.go:64] FLAG: --registry-qps="5"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341952    2568 flags.go:64] FLAG: --reserved-cpus=""
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341956    2568 flags.go:64] FLAG: --reserved-memory=""
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341962    2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341967    2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 20 16:45:19.350873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341971    2568 flags.go:64] FLAG: --rotate-certificates="false"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341976    2568 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341980    2568 flags.go:64] FLAG: --runonce="false"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341985    2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341990    2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.341995    2568 flags.go:64] FLAG: --seccomp-default="false"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342000    2568 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342005    2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342010    2568 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342014    2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342019    2568 flags.go:64] FLAG: --storage-driver-password="root"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342023    2568 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342027    2568 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342032    2568 flags.go:64] FLAG: --storage-driver-user="root"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342036    2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342041    2568 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342046    2568 flags.go:64] FLAG: --system-cgroups=""
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342050    2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342059    2568 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342065    2568 flags.go:64] FLAG: --tls-cert-file=""
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342071    2568 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342077    2568 flags.go:64] FLAG: --tls-min-version=""
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342081    2568 flags.go:64] FLAG: --tls-private-key-file=""
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342086    2568 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342090    2568 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 20 16:45:19.351570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342096    2568 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342100    2568 flags.go:64] FLAG: --v="2"
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342106    2568 flags.go:64] FLAG: --version="false"
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342112    2568 flags.go:64] FLAG: --vmodule=""
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342118    2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342124    2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342392    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342401    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342406    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342411    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342416    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342420    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342425    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342430    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342434    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342439    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342443    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342447    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342451    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342455    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342460    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 16:45:19.352206 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342464    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342468    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342472    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342498    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342504    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342510    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342517    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342521    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342525    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342530    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342534    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342538    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342550    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342555    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342559    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342563    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342567    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342572    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342576    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342580    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 16:45:19.352748 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342585    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342591    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342597    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342602    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342606    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342610    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342614    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342620    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342625    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342628    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342633    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342637    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342641    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342645    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342649    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342653    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342657    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342663    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342669    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 16:45:19.353237 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342673    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342677    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342681    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342685    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342690    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342694    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342707    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342712    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342716    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342721    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342725    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342729    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342733    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342737    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342742    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342746    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342750    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342756    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342761    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 16:45:19.353699 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342766    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342770    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342774    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342778    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342782    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342786    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342790    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342794    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342798    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342803    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342806    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342812    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342817    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342821    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342825    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342830    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342834    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342838    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342844    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 16:45:19.354215 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342849    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342861    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342866    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342870    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342875    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342879    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.342883    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.342891    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.351290    2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.351305    2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.351326    2568 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.351338    2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351394    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351398    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351404    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 16:45:19.354688 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351409    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351414    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351417    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351419    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351422    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351425    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351427    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351429    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351432    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351434    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351437    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351439    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351441    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351444    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351446    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351448    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351451    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351453    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351455    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351458    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 16:45:19.355187 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351460    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351463    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351467    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351470    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351473    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351477    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351482    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351487    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351491    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351495    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351497    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351500    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351503    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351505    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351508    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351511    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351513    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351516    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351519    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 16:45:19.355670 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351523    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351525    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351528    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351532    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351536    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351539    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351541    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351544    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351546    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351549    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351552    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351557    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351561    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351565    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351570    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351572    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351575    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351578    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351580    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 16:45:19.356117 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351583    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351586    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351588    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351591    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351593    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351596    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351598    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351600    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351603    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351606    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351608    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351610    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351613    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351615    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351618    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351620    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351622    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351624    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351628    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351630    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 16:45:19.356595 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351634    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351638    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351642    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351646    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351649    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351651    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351654    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351656    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351658    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351661    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351663    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351665    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351668    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351670    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351672    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351675    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351677    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 16:45:19.357062 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351680    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.351685    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351809    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351814    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351817    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351820    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351823    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351825    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351827    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351829    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351832    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351834    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351836    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351839    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351849    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 16:45:19.357483 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351852    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351854    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351856    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351858    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351861    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351863    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351867    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351870    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351874    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351878    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351881    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351884    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351886    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351889    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351891    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351894    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351896    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351898    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351901    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351903    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 16:45:19.357839 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351905    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351908    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351911    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351916    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351918    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351920    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351922    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351925    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351927    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351929    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351931    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351934    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351942    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351947    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351951    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351955    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351959    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351961    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351964    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 16:45:19.358389 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351966    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351968    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351971    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351973    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351975    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351977    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351982    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351985    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351988    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351991    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351993    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351996    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.351999    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352002    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352004    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352007    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352009    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352012    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352014    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 16:45:19.358889 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352017    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352019    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352022    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352027    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352033    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352037    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352040    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352044    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352047    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352049    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352051    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352054    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352056    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352059    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352061    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352063    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352066    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352068    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352070    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352073    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 16:45:19.359477 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352075    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 16:45:19.360087 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352078    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 16:45:19.360087 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352080    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 16:45:19.360087 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352082    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 16:45:19.360087 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352085    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 16:45:19.360087 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352087    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 16:45:19.360087 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352089    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 16:45:19.360087 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:19.352092    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 16:45:19.360087 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.352097    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 16:45:19.360087 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.352232    2568 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 20 16:45:19.360087 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.355296    2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 20 16:45:19.360087 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.356524    2568 server.go:1019] "Starting client certificate rotation"
Jul 20 16:45:19.360087 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.356626    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 16:45:19.360087 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.356666    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 16:45:19.360435 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.358697    2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 16:45:19.361188 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.361173    2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 16:45:19.371006 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.370985    2568 log.go:25] "Validated CRI v1 runtime API"
Jul 20 16:45:19.373191 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.373177    2568 log.go:25] "Validated CRI v1 image API"
Jul 20 16:45:19.374069 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.374055    2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 20 16:45:19.378187 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.378169    2568 fs.go:135] Filesystem UUIDs: map[270a611e-e6b8-4ebc-9654-45bf264a86e8:/dev/nvme0n1p3 3068be71-b216-4d12-8b22-5db7deaaafcb:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 20 16:45:19.378245 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.378187    2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 20 16:45:19.381358 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.381329    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 16:45:19.384729 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.384625    2568 manager.go:216] Machine: {Timestamp:2026-07-20 16:45:19.383353126 +0000 UTC m=+0.281302101 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3135847 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2783963a1b20b4b95d83b0a5a5ca62 SystemUUID:ec278396-3a1b-20b4-b95d-83b0a5a5ca62 BootID:08c078e5-fe68-4b6c-969f-082642fe73c4 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:bb:1d:7e:d3:21 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:bb:1d:7e:d3:21 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:86:a1:f2:71:13 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 20 16:45:19.384729 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.384721    2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 20 16:45:19.384868 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.384823    2568 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 20 16:45:19.387434 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.387405    2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 20 16:45:19.387565 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.387437    2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-8.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 20 16:45:19.388066 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.388057    2568 topology_manager.go:138] "Creating topology manager with none policy"
Jul 20 16:45:19.388097 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.388069    2568 container_manager_linux.go:309] "Creating device plugin manager"
Jul 20 16:45:19.388097 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.388075    2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 16:45:19.388673 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.388664    2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 16:45:19.389992 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.389982    2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 20 16:45:19.393652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.393641    2568 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 16:45:19.393767 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.393758    2568 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 20 16:45:19.393804 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.393787    2568 kubelet.go:486] "Attempting to sync node with API server"
Jul 20 16:45:19.393804 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.393803    2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 20 16:45:19.393899 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.393819    2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 20 16:45:19.393899 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.393830    2568 kubelet.go:398] "Adding apiserver pod source"
Jul 20 16:45:19.393899 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.393842    2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 20 16:45:19.394261 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.394249    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 16:45:19.394312 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.394267    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 16:45:19.397160 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.397132    2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 20 16:45:19.398935 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.398919    2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 20 16:45:19.398983 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.398951    2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 20 16:45:19.400101 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400088    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 20 16:45:19.400165 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400112    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 20 16:45:19.400165 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400119    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 20 16:45:19.400165 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400125    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 20 16:45:19.400165 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400134    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 20 16:45:19.400165 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400139    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 20 16:45:19.400165 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400165    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 20 16:45:19.400329 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400171    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 20 16:45:19.400329 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400177    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 20 16:45:19.400329 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400182    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 20 16:45:19.400329 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400190    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 20 16:45:19.400329 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400196    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 20 16:45:19.400793 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400784    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 20 16:45:19.400793 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.400793    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 20 16:45:19.403852 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.403819    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 20 16:45:19.403923 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.403822    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-8.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 20 16:45:19.403923 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.403911    2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-8.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 20 16:45:19.404087 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.404076    2568 server.go:1268] "Started kubelet"
Jul 20 16:45:19.404200 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.404178    2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 20 16:45:19.404796 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.404613    2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 20 16:45:19.404947 ip-10-0-128-8 systemd[1]: Started Kubernetes Kubelet.
Jul 20 16:45:19.405053 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.404862    2568 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 20 16:45:19.406186 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.406174    2568 server.go:310] "Adding debug handlers to kubelet server"
Jul 20 16:45:19.406956 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.406939    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m4chk"
Jul 20 16:45:19.407992 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.407964    2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 20 16:45:19.409951 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.409933    2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 20 16:45:19.413586 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.413555    2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 20 16:45:19.413669 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.413555    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 20 16:45:19.413726 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.413682    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-8.ec2.internal\" not found"
Jul 20 16:45:19.413958 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.412930    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-8.ec2.internal.18c40d3fa858c140  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-8.ec2.internal,UID:ip-10-0-128-8.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-8.ec2.internal,},FirstTimestamp:2026-07-20 16:45:19.404048704 +0000 UTC m=+0.301997679,LastTimestamp:2026-07-20 16:45:19.404048704 +0000 UTC m=+0.301997679,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-8.ec2.internal,}"
Jul 20 16:45:19.414126 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.414111    2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 20 16:45:19.414228 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.414116    2568 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 20 16:45:19.414228 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.414165    2568 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 20 16:45:19.414334 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.414299    2568 reconstruct.go:97] "Volume reconstruction finished"
Jul 20 16:45:19.414334 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.414308    2568 reconciler.go:29] "Reconciler: start to sync state"
Jul 20 16:45:19.414819 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.414801    2568 factory.go:55] Registering systemd factory
Jul 20 16:45:19.414819 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.414818    2568 factory.go:223] Registration of the systemd container factory successfully
Jul 20 16:45:19.414939 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.414846    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m4chk"
Jul 20 16:45:19.415103 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.415090    2568 factory.go:153] Registering CRI-O factory
Jul 20 16:45:19.415168 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.415107    2568 factory.go:223] Registration of the crio container factory successfully
Jul 20 16:45:19.415229 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.415191    2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 20 16:45:19.415229 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.415211    2568 factory.go:103] Registering Raw factory
Jul 20 16:45:19.415316 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.415231    2568 manager.go:1196] Started watching for new ooms in manager
Jul 20 16:45:19.415747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.415732    2568 manager.go:349] Starting recovery of all containers
Jul 20 16:45:19.418294 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.418266    2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-8.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 20 16:45:19.418429 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.418399    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 20 16:45:19.427843 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.427685    2568 manager.go:354] Recovery completed
Jul 20 16:45:19.431924 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.431912    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:45:19.434038 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.434021    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:45:19.434097 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.434049    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:45:19.434097 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.434061    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:45:19.434596 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.434580    2568 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 20 16:45:19.434650 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.434596    2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 20 16:45:19.434650 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.434584    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:45:19.434650 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.434615    2568 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 16:45:19.435278 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.435264    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:45:19.435310 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.435292    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:45:19.435310 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.435304    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:45:19.436727 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.436713    2568 policy_none.go:49] "None policy: Start"
Jul 20 16:45:19.436770 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.436732    2568 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 20 16:45:19.436770 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.436745    2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 20 16:45:19.438881 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.438869    2568 policy_none.go:47] "Start"
Jul 20 16:45:19.472698 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.472678    2568 manager.go:337] "Starting Device Plugin manager"
Jul 20 16:45:19.472789 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.472708    2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 20 16:45:19.472789 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.472720    2568 server.go:87] "Starting device plugin registration server"
Jul 20 16:45:19.472973 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.472960    2568 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 20 16:45:19.473022 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.472974    2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 20 16:45:19.473077 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.473066    2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 20 16:45:19.473536 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.473160    2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 20 16:45:19.473536 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.473173    2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 20 16:45:19.473666 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.473623    2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 20 16:45:19.473666 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.473665    2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-8.ec2.internal\" not found"
Jul 20 16:45:19.538196 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.538169    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 20 16:45:19.540409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.539497    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 20 16:45:19.540409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.539527    2568 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 20 16:45:19.540409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.539548    2568 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 20 16:45:19.540409 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.539578    2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 20 16:45:19.542000 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.541983    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 16:45:19.573681 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.573655    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:45:19.574459 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.574435    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:45:19.574561 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.574472    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:45:19.574561 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.574489    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:45:19.574561 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.574514    2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.579687 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.579674    2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.579727 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.579694    2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-8.ec2.internal\": node \"ip-10-0-128-8.ec2.internal\" not found"
Jul 20 16:45:19.593481 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.593439    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-8.ec2.internal\" not found"
Jul 20 16:45:19.640256 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.640235    2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-8.ec2.internal"]
Jul 20 16:45:19.640322 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.640292    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:45:19.641584 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.641570    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:45:19.641702 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.641597    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:45:19.641702 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.641607    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:45:19.643887 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.643876    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:45:19.643995 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.643981    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.644047 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.644008    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:45:19.644490 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.644477    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:45:19.644557 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.644500    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:45:19.644557 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.644513    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:45:19.644640 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.644575    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:45:19.644640 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.644591    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:45:19.644640 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.644604    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:45:19.646692 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.646676    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.646775 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.646698    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:45:19.647456 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.647440    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:45:19.647456 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.647459    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:45:19.647598 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.647470    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:45:19.670720 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.670703    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-8.ec2.internal\" not found" node="ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.674262 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.674244    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-8.ec2.internal\" not found" node="ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.693954 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.693912    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-8.ec2.internal\" not found"
Jul 20 16:45:19.715246 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.715226    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/feab54f1096556ef0bdc91b7b84f5a56-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal\" (UID: \"feab54f1096556ef0bdc91b7b84f5a56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.715364 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.715250    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/feab54f1096556ef0bdc91b7b84f5a56-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal\" (UID: \"feab54f1096556ef0bdc91b7b84f5a56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.715364 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.715268    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e1dddf4097058c612737a8bf9968fc66-config\") pod \"kube-apiserver-proxy-ip-10-0-128-8.ec2.internal\" (UID: \"e1dddf4097058c612737a8bf9968fc66\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.794321 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.794289    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-8.ec2.internal\" not found"
Jul 20 16:45:19.815594 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.815565    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/feab54f1096556ef0bdc91b7b84f5a56-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal\" (UID: \"feab54f1096556ef0bdc91b7b84f5a56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.815654 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.815601    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/feab54f1096556ef0bdc91b7b84f5a56-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal\" (UID: \"feab54f1096556ef0bdc91b7b84f5a56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.815654 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.815619    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e1dddf4097058c612737a8bf9968fc66-config\") pod \"kube-apiserver-proxy-ip-10-0-128-8.ec2.internal\" (UID: \"e1dddf4097058c612737a8bf9968fc66\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.815654 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.815648    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/feab54f1096556ef0bdc91b7b84f5a56-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal\" (UID: \"feab54f1096556ef0bdc91b7b84f5a56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.815743 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.815652    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/feab54f1096556ef0bdc91b7b84f5a56-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal\" (UID: \"feab54f1096556ef0bdc91b7b84f5a56\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.815743 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.815680    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e1dddf4097058c612737a8bf9968fc66-config\") pod \"kube-apiserver-proxy-ip-10-0-128-8.ec2.internal\" (UID: \"e1dddf4097058c612737a8bf9968fc66\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.894832 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.894771    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-8.ec2.internal\" not found"
Jul 20 16:45:19.974354 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.974312    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.977472 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:19.977450    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:19.995759 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:19.995738    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-8.ec2.internal\" not found"
Jul 20 16:45:20.096397 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.096351    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-8.ec2.internal\" not found"
Jul 20 16:45:20.196912 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.196822    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-8.ec2.internal\" not found"
Jul 20 16:45:20.297415 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.297385    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-8.ec2.internal\" not found"
Jul 20 16:45:20.307626 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.307609    2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 16:45:20.314361 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.314344    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:20.325121 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.325101    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 16:45:20.326597 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.326584    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-8.ec2.internal"
Jul 20 16:45:20.334949 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.334931    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 16:45:20.356036 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.356008    2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 20 16:45:20.356202 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.356181    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 16:45:20.356260 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.356197    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 16:45:20.394347 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.394315    2568 apiserver.go:52] "Watching apiserver"
Jul 20 16:45:20.409290 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.409263    2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 20 16:45:20.409581 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.409562    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-m28fr","openshift-network-diagnostics/network-check-target-ql4vb","openshift-ovn-kubernetes/ovnkube-node-44gfk","kube-system/global-pull-secret-syncer-ntr6q","kube-system/konnectivity-agent-8phrk","kube-system/kube-apiserver-proxy-ip-10-0-128-8.ec2.internal","openshift-cluster-node-tuning-operator/tuned-wbchd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal","openshift-multus/multus-additional-cni-plugins-f844c","openshift-multus/multus-brgfz","openshift-network-operator/iptables-alerter-fpj7r","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr","openshift-dns/node-resolver-6g5w2","openshift-image-registry/node-ca-sfzwq"]
Jul 20 16:45:20.412085 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.412071    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.414117 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.414087    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 20 16:45:20.414215 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.414201    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:20.414280 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.414258    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:20.414632 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.414611    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 20 16:45:20.414716 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.414685    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 20 16:45:20.414799 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.414783    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 20 16:45:20.414859 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.414829    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8ghx2\""
Jul 20 16:45:20.414930 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.414913    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 20 16:45:20.414975 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.414913    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 20 16:45:20.416524 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.416504    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.418662 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.418643    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 20 16:45:20.418801 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.418783    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:20.418907 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.418832    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:20.418907 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.418870    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff628bb3-d193-412f-bae5-c23349f488da-system-cni-dir\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.419032 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.418910    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 20 16:45:20.419032 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.418920    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 20 16:45:20.419032 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.418911    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff628bb3-d193-412f-bae5-c23349f488da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.419032 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.418979    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-kubelet\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419032 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.418995    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-systemd-units\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419032 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419022    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j6745\""
Jul 20 16:45:20.419032 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419025    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-cni-bin\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419049    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ff628bb3-d193-412f-bae5-c23349f488da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.419296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419065    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ff628bb3-d193-412f-bae5-c23349f488da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.419296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419081    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtcj7\" (UniqueName: \"kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7\") pod \"network-check-target-ql4vb\" (UID: \"ef355774-d5d7-4b50-9643-9ba08afd080d\") " pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:20.419296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419101    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/75b664df-5e35-4825-a3ae-e8310a71b97e-ovn-node-metrics-cert\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419137    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff628bb3-d193-412f-bae5-c23349f488da-cnibin\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.419296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419177    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff628bb3-d193-412f-bae5-c23349f488da-cni-binary-copy\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.419296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419195    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-run-openvswitch\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419208    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9m42\" (UniqueName: \"kubernetes.io/projected/75b664df-5e35-4825-a3ae-e8310a71b97e-kube-api-access-n9m42\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419212    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 20 16:45:20.419296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419221    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-cni-netd\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419244    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/75b664df-5e35-4825-a3ae-e8310a71b97e-ovnkube-config\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419270    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/75b664df-5e35-4825-a3ae-e8310a71b97e-env-overrides\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419292    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ff628bb3-d193-412f-bae5-c23349f488da-os-release\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419321    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65gcd\" (UniqueName: \"kubernetes.io/projected/ff628bb3-d193-412f-bae5-c23349f488da-kube-api-access-65gcd\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419344    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/75b664df-5e35-4825-a3ae-e8310a71b97e-ovnkube-script-lib\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419361    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-slash\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419375    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-run-netns\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419388    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-node-log\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419395    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419407    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-run-ovn-kubernetes\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419411    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419433    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-run-systemd\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419459    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-var-lib-openvswitch\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419473    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-etc-openvswitch\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419486    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-run-ovn\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419513    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-log-socket\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419540    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419615    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 16:40:20 +0000 UTC" deadline="2028-04-06 00:50:09.39108288 +0000 UTC"
Jul 20 16:45:20.419713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.419633    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15008h4m48.971452328s"
Jul 20 16:45:20.420905 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.420885    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6g5w2"
Jul 20 16:45:20.421373 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.421358    2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 16:45:20.423076 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.423062    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 20 16:45:20.423173 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.423092    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.423234 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.423200    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 20 16:45:20.423425 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.423410    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f75sm\""
Jul 20 16:45:20.425374 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.425357    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:20.425433 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.425417    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:20.426792 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.426775    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-npdc4\""
Jul 20 16:45:20.426881 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.426853    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 20 16:45:20.426881 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.426861    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 16:45:20.427587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.427570    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.430792 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.430775    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 20 16:45:20.430862 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.430776    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 20 16:45:20.431053 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.431040    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nvr7h\""
Jul 20 16:45:20.431319 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.431309    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fpj7r"
Jul 20 16:45:20.433626 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.433611    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.433741 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.433726    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 20 16:45:20.434101 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.434088    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xhg7p\""
Jul 20 16:45:20.434357 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.434344    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 20 16:45:20.435889 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.435875    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8phrk"
Jul 20 16:45:20.436610 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.436593    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 20 16:45:20.437270 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.437256    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 20 16:45:20.437323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.437272    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-s8n4p\""
Jul 20 16:45:20.438212 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.438194    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sfzwq"
Jul 20 16:45:20.450683 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.450637    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 20 16:45:20.450857 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.450841    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 20 16:45:20.450926 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.450843    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 20 16:45:20.451027 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.450997    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tv7gn\""
Jul 20 16:45:20.451088 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.451073    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 20 16:45:20.461375 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.461356    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w2gzp\""
Jul 20 16:45:20.461537 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.461512    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wn6jj"
Jul 20 16:45:20.461604 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.461567    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 20 16:45:20.461604 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.461590    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 20 16:45:20.470427 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.470413    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 20 16:45:20.477081 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.477019    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wn6jj"
Jul 20 16:45:20.515095 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.515073    2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 20 16:45:20.520292 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520048    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/798679f7-4546-4546-bc3d-c12693574272-hosts-file\") pod \"node-resolver-6g5w2\" (UID: \"798679f7-4546-4546-bc3d-c12693574272\") " pod="openshift-dns/node-resolver-6g5w2"
Jul 20 16:45:20.520292 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520080    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-hostroot\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.520292 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520102    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-sysctl-conf\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.520292 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520123    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1e4a352-901a-4050-b0c4-5e63ec823420-kubelet-config\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:20.520292 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520159    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.520292 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520182    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-sys-fs\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.520292 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520202    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-registration-dir\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.520292 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520228    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46c16140-8350-4ab4-9e33-b92a7fb16f1e-host\") pod \"node-ca-sfzwq\" (UID: \"46c16140-8350-4ab4-9e33-b92a7fb16f1e\") " pod="openshift-image-registry/node-ca-sfzwq"
Jul 20 16:45:20.520292 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520260    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-run-systemd\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.520292 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520282    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-run-ovn\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.520747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520302    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-log-socket\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.520747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520321    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.520747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520342    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff628bb3-d193-412f-bae5-c23349f488da-system-cni-dir\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.520747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520378    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff628bb3-d193-412f-bae5-c23349f488da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.520747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520396    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-run\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.520747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520426    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-device-dir\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.520747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520446    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-sysctl-d\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.520747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520469    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-kubelet\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.520747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520489    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-systemd-units\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.520747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520509    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-cnibin\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.520747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520531    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-multus-socket-dir-parent\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.520747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520558    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-lib-modules\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.520747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520608    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-682pd\" (UniqueName: \"kubernetes.io/projected/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-kube-api-access-682pd\") pod \"network-metrics-daemon-m28fr\" (UID: \"87e02cfa-5d5b-4020-ac07-c4cf81f3e099\") " pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520772    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b3bec34-0628-43fe-9211-2203141bd83c-konnectivity-ca\") pod \"konnectivity-agent-8phrk\" (UID: \"9b3bec34-0628-43fe-9211-2203141bd83c\") " pod="kube-system/konnectivity-agent-8phrk"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520814    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff628bb3-d193-412f-bae5-c23349f488da-cni-binary-copy\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520837    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-os-release\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520859    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-var-lib-cni-bin\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520882    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/826f2d27-7719-4185-8e92-d020c83a8409-multus-daemon-config\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520905    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-host\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.520939    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-tmp\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521003    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-systemd-units\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521003    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-kubelet\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521007    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9m42\" (UniqueName: \"kubernetes.io/projected/75b664df-5e35-4825-a3ae-e8310a71b97e-kube-api-access-n9m42\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521057    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/826f2d27-7719-4185-8e92-d020c83a8409-cni-binary-copy\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521083    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-run-k8s-cni-cncf-io\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521084    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff628bb3-d193-412f-bae5-c23349f488da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521130    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-tuned\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521129    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-log-socket\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521176    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/75b664df-5e35-4825-a3ae-e8310a71b97e-ovnkube-config\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.521323 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521217    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521219    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-run-systemd\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521240    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/75b664df-5e35-4825-a3ae-e8310a71b97e-env-overrides\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521320    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ff628bb3-d193-412f-bae5-c23349f488da-os-release\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521368    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-var-lib-kubelet\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521414    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgfgq\" (UniqueName: \"kubernetes.io/projected/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-kube-api-access-cgfgq\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521456    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnjkd\" (UniqueName: \"kubernetes.io/projected/46c16140-8350-4ab4-9e33-b92a7fb16f1e-kube-api-access-gnjkd\") pod \"node-ca-sfzwq\" (UID: \"46c16140-8350-4ab4-9e33-b92a7fb16f1e\") " pod="openshift-image-registry/node-ca-sfzwq"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521520    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff628bb3-d193-412f-bae5-c23349f488da-system-cni-dir\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521534    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ff628bb3-d193-412f-bae5-c23349f488da-os-release\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521558    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-run-multus-certs\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521591    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/75b664df-5e35-4825-a3ae-e8310a71b97e-ovnkube-script-lib\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521627    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-multus-conf-dir\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521700    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/75b664df-5e35-4825-a3ae-e8310a71b97e-env-overrides\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521723    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-run-ovn\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521725    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff628bb3-d193-412f-bae5-c23349f488da-cni-binary-copy\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521725    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-var-lib-kubelet\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521768    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-slash\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521772    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/75b664df-5e35-4825-a3ae-e8310a71b97e-ovnkube-config\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521792    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-run-netns\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521825    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-node-log\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521827    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-run-netns\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521842    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-slash\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521869    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-run-ovn-kubernetes\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521873    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-node-log\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521898    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r9nt\" (UniqueName: \"kubernetes.io/projected/3668c731-b9db-4e02-a8e7-16357f247006-kube-api-access-5r9nt\") pod \"iptables-alerter-fpj7r\" (UID: \"3668c731-b9db-4e02-a8e7-16357f247006\") " pod="openshift-network-operator/iptables-alerter-fpj7r"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521944    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1e4a352-901a-4050-b0c4-5e63ec823420-dbus\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521979    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-run-ovn-kubernetes\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.521996    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46c16140-8350-4ab4-9e33-b92a7fb16f1e-serviceca\") pod \"node-ca-sfzwq\" (UID: \"46c16140-8350-4ab4-9e33-b92a7fb16f1e\") " pod="openshift-image-registry/node-ca-sfzwq"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522032    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-var-lib-openvswitch\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522073    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-etc-openvswitch\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522079    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-var-lib-openvswitch\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522110    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ff628bb3-d193-412f-bae5-c23349f488da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522128    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3668c731-b9db-4e02-a8e7-16357f247006-host-slash\") pod \"iptables-alerter-fpj7r\" (UID: \"3668c731-b9db-4e02-a8e7-16357f247006\") " pod="openshift-network-operator/iptables-alerter-fpj7r"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522176    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs\") pod \"network-metrics-daemon-m28fr\" (UID: \"87e02cfa-5d5b-4020-ac07-c4cf81f3e099\") " pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:20.522869 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522175    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/75b664df-5e35-4825-a3ae-e8310a71b97e-ovnkube-script-lib\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522192    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-sysconfig\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522193    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-etc-openvswitch\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522218    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-cni-bin\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522237    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ff628bb3-d193-412f-bae5-c23349f488da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522265    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-cni-bin\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522261    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtcj7\" (UniqueName: \"kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7\") pod \"network-check-target-ql4vb\" (UID: \"ef355774-d5d7-4b50-9643-9ba08afd080d\") " pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522306    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/798679f7-4546-4546-bc3d-c12693574272-tmp-dir\") pod \"node-resolver-6g5w2\" (UID: \"798679f7-4546-4546-bc3d-c12693574272\") " pod="openshift-dns/node-resolver-6g5w2"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522331    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm49r\" (UniqueName: \"kubernetes.io/projected/826f2d27-7719-4185-8e92-d020c83a8409-kube-api-access-cm49r\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522356    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522380    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-socket-dir\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522422    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/75b664df-5e35-4825-a3ae-e8310a71b97e-ovn-node-metrics-cert\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522446    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff628bb3-d193-412f-bae5-c23349f488da-cnibin\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522483    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zps72\" (UniqueName: \"kubernetes.io/projected/798679f7-4546-4546-bc3d-c12693574272-kube-api-access-zps72\") pod \"node-resolver-6g5w2\" (UID: \"798679f7-4546-4546-bc3d-c12693574272\") " pod="openshift-dns/node-resolver-6g5w2"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522533    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-multus-cni-dir\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522557    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-var-lib-cni-multus\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522579    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-etc-kubernetes\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.523431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522600    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-systemd\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522612    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ff628bb3-d193-412f-bae5-c23349f488da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522626    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-run-openvswitch\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522666    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-system-cni-dir\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522682    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-run-openvswitch\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522729    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ff628bb3-d193-412f-bae5-c23349f488da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522755    2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522761    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-modprobe-d\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522791    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff628bb3-d193-412f-bae5-c23349f488da-cnibin\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522837    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3668c731-b9db-4e02-a8e7-16357f247006-iptables-alerter-script\") pod \"iptables-alerter-fpj7r\" (UID: \"3668c731-b9db-4e02-a8e7-16357f247006\") " pod="openshift-network-operator/iptables-alerter-fpj7r"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522867    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-cni-netd\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522905    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-run-netns\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522949    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-etc-selinux\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.522976    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhrrk\" (UniqueName: \"kubernetes.io/projected/6d79d643-3794-4ea2-98a4-7aabcdda7572-kube-api-access-mhrrk\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.523014    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b3bec34-0628-43fe-9211-2203141bd83c-agent-certs\") pod \"konnectivity-agent-8phrk\" (UID: \"9b3bec34-0628-43fe-9211-2203141bd83c\") " pod="kube-system/konnectivity-agent-8phrk"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.523063    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65gcd\" (UniqueName: \"kubernetes.io/projected/ff628bb3-d193-412f-bae5-c23349f488da-kube-api-access-65gcd\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.523091    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-kubernetes\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.524235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.523117    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-sys\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.524743 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.523211    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/75b664df-5e35-4825-a3ae-e8310a71b97e-host-cni-netd\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.525555 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.525537    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/75b664df-5e35-4825-a3ae-e8310a71b97e-ovn-node-metrics-cert\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.528993 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.528972    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:45:20.529086 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.529000    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:45:20.529086 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.529013    2568 projected.go:196] Error preparing data for projected volume kube-api-access-qtcj7 for pod openshift-network-diagnostics/network-check-target-ql4vb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:20.529211 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.529097    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7 podName:ef355774-d5d7-4b50-9643-9ba08afd080d nodeName:}" failed. No retries permitted until 2026-07-20 16:45:21.029061574 +0000 UTC m=+1.927010560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qtcj7" (UniqueName: "kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7") pod "network-check-target-ql4vb" (UID: "ef355774-d5d7-4b50-9643-9ba08afd080d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:20.531191 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.531171    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9m42\" (UniqueName: \"kubernetes.io/projected/75b664df-5e35-4825-a3ae-e8310a71b97e-kube-api-access-n9m42\") pod \"ovnkube-node-44gfk\" (UID: \"75b664df-5e35-4825-a3ae-e8310a71b97e\") " pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.534758 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.532187    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65gcd\" (UniqueName: \"kubernetes.io/projected/ff628bb3-d193-412f-bae5-c23349f488da-kube-api-access-65gcd\") pod \"multus-additional-cni-plugins-f844c\" (UID: \"ff628bb3-d193-412f-bae5-c23349f488da\") " pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.623949 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.623919    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-run-netns\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.623949 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.623947    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-etc-selinux\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.624170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.623962    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhrrk\" (UniqueName: \"kubernetes.io/projected/6d79d643-3794-4ea2-98a4-7aabcdda7572-kube-api-access-mhrrk\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.624170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.623977    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b3bec34-0628-43fe-9211-2203141bd83c-agent-certs\") pod \"konnectivity-agent-8phrk\" (UID: \"9b3bec34-0628-43fe-9211-2203141bd83c\") " pod="kube-system/konnectivity-agent-8phrk"
Jul 20 16:45:20.624170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.623997    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-kubernetes\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.624170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624030    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-run-netns\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.624170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624044    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-sys\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.624170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624054    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-kubernetes\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.624170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624078    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/798679f7-4546-4546-bc3d-c12693574272-hosts-file\") pod \"node-resolver-6g5w2\" (UID: \"798679f7-4546-4546-bc3d-c12693574272\") " pod="openshift-dns/node-resolver-6g5w2"
Jul 20 16:45:20.624170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624102    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-sys\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.624170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624104    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-etc-selinux\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.624170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624103    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-hostroot\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.624170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624166    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/798679f7-4546-4546-bc3d-c12693574272-hosts-file\") pod \"node-resolver-6g5w2\" (UID: \"798679f7-4546-4546-bc3d-c12693574272\") " pod="openshift-dns/node-resolver-6g5w2"
Jul 20 16:45:20.624170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624171    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-sysctl-conf\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624137    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-hostroot\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624200    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1e4a352-901a-4050-b0c4-5e63ec823420-kubelet-config\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624253    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1e4a352-901a-4050-b0c4-5e63ec823420-kubelet-config\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624254    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624256    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-sysctl-conf\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624293    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-sys-fs\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624307    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624316    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-registration-dir\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624344    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46c16140-8350-4ab4-9e33-b92a7fb16f1e-host\") pod \"node-ca-sfzwq\" (UID: \"46c16140-8350-4ab4-9e33-b92a7fb16f1e\") " pod="openshift-image-registry/node-ca-sfzwq"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624347    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-sys-fs\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624355    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-registration-dir\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624371    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-run\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624384    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46c16140-8350-4ab4-9e33-b92a7fb16f1e-host\") pod \"node-ca-sfzwq\" (UID: \"46c16140-8350-4ab4-9e33-b92a7fb16f1e\") " pod="openshift-image-registry/node-ca-sfzwq"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624393    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-device-dir\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624430    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-device-dir\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624415    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-sysctl-d\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.624753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624443    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-run\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624467    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-cnibin\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624485    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-multus-socket-dir-parent\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624499    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-lib-modules\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624511    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-sysctl-d\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624529    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-682pd\" (UniqueName: \"kubernetes.io/projected/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-kube-api-access-682pd\") pod \"network-metrics-daemon-m28fr\" (UID: \"87e02cfa-5d5b-4020-ac07-c4cf81f3e099\") " pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624541    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-cnibin\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624551    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b3bec34-0628-43fe-9211-2203141bd83c-konnectivity-ca\") pod \"konnectivity-agent-8phrk\" (UID: \"9b3bec34-0628-43fe-9211-2203141bd83c\") " pod="kube-system/konnectivity-agent-8phrk"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624555    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-multus-socket-dir-parent\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624566    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-os-release\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624588    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-var-lib-cni-bin\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624610    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/826f2d27-7719-4185-8e92-d020c83a8409-multus-daemon-config\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624645    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-host\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624655    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-os-release\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624664    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-tmp\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624683    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/826f2d27-7719-4185-8e92-d020c83a8409-cni-binary-copy\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624698    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-run-k8s-cni-cncf-io\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624712    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-tuned\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.625409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624728    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-var-lib-kubelet\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624784    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgfgq\" (UniqueName: \"kubernetes.io/projected/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-kube-api-access-cgfgq\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624788    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-var-lib-cni-bin\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624805    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnjkd\" (UniqueName: \"kubernetes.io/projected/46c16140-8350-4ab4-9e33-b92a7fb16f1e-kube-api-access-gnjkd\") pod \"node-ca-sfzwq\" (UID: \"46c16140-8350-4ab4-9e33-b92a7fb16f1e\") " pod="openshift-image-registry/node-ca-sfzwq"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624857    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-run-multus-certs\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624885    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-multus-conf-dir\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624911    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-var-lib-kubelet\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624938    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r9nt\" (UniqueName: \"kubernetes.io/projected/3668c731-b9db-4e02-a8e7-16357f247006-kube-api-access-5r9nt\") pod \"iptables-alerter-fpj7r\" (UID: \"3668c731-b9db-4e02-a8e7-16357f247006\") " pod="openshift-network-operator/iptables-alerter-fpj7r"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624961    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1e4a352-901a-4050-b0c4-5e63ec823420-dbus\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624984    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46c16140-8350-4ab4-9e33-b92a7fb16f1e-serviceca\") pod \"node-ca-sfzwq\" (UID: \"46c16140-8350-4ab4-9e33-b92a7fb16f1e\") " pod="openshift-image-registry/node-ca-sfzwq"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625009    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3668c731-b9db-4e02-a8e7-16357f247006-host-slash\") pod \"iptables-alerter-fpj7r\" (UID: \"3668c731-b9db-4e02-a8e7-16357f247006\") " pod="openshift-network-operator/iptables-alerter-fpj7r"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625031    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs\") pod \"network-metrics-daemon-m28fr\" (UID: \"87e02cfa-5d5b-4020-ac07-c4cf81f3e099\") " pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625054    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-sysconfig\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625065    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-run-k8s-cni-cncf-io\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625092    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/798679f7-4546-4546-bc3d-c12693574272-tmp-dir\") pod \"node-resolver-6g5w2\" (UID: \"798679f7-4546-4546-bc3d-c12693574272\") " pod="openshift-dns/node-resolver-6g5w2"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625113    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b3bec34-0628-43fe-9211-2203141bd83c-konnectivity-ca\") pod \"konnectivity-agent-8phrk\" (UID: \"9b3bec34-0628-43fe-9211-2203141bd83c\") " pod="kube-system/konnectivity-agent-8phrk"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625115    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm49r\" (UniqueName: \"kubernetes.io/projected/826f2d27-7719-4185-8e92-d020c83a8409-kube-api-access-cm49r\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.626258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625183    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625210    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-socket-dir\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625254    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3668c731-b9db-4e02-a8e7-16357f247006-host-slash\") pod \"iptables-alerter-fpj7r\" (UID: \"3668c731-b9db-4e02-a8e7-16357f247006\") " pod="openshift-network-operator/iptables-alerter-fpj7r"
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.625327    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625328    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d79d643-3794-4ea2-98a4-7aabcdda7572-socket-dir\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625135    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-var-lib-kubelet\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.625392    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs podName:87e02cfa-5d5b-4020-ac07-c4cf81f3e099 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:21.125374821 +0000 UTC m=+2.023323785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs") pod "network-metrics-daemon-m28fr" (UID: "87e02cfa-5d5b-4020-ac07-c4cf81f3e099") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625437    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/826f2d27-7719-4185-8e92-d020c83a8409-cni-binary-copy\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.624609    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-lib-modules\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625182    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/826f2d27-7719-4185-8e92-d020c83a8409-multus-daemon-config\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625500    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-host\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625529    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-var-lib-kubelet\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.625441    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:20.625600    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret podName:e1e4a352-901a-4050-b0c4-5e63ec823420 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:21.125585102 +0000 UTC m=+2.023534064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret") pod "global-pull-secret-syncer-ntr6q" (UID: "e1e4a352-901a-4050-b0c4-5e63ec823420") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625624    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-sysconfig\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625649    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-run-multus-certs\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625765    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1e4a352-901a-4050-b0c4-5e63ec823420-dbus\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:20.626798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625821    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-multus-conf-dir\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625862    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zps72\" (UniqueName: \"kubernetes.io/projected/798679f7-4546-4546-bc3d-c12693574272-kube-api-access-zps72\") pod \"node-resolver-6g5w2\" (UID: \"798679f7-4546-4546-bc3d-c12693574272\") " pod="openshift-dns/node-resolver-6g5w2"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625896    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-multus-cni-dir\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625937    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/798679f7-4546-4546-bc3d-c12693574272-tmp-dir\") pod \"node-resolver-6g5w2\" (UID: \"798679f7-4546-4546-bc3d-c12693574272\") " pod="openshift-dns/node-resolver-6g5w2"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625946    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-var-lib-cni-multus\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.625989    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46c16140-8350-4ab4-9e33-b92a7fb16f1e-serviceca\") pod \"node-ca-sfzwq\" (UID: \"46c16140-8350-4ab4-9e33-b92a7fb16f1e\") " pod="openshift-image-registry/node-ca-sfzwq"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.626004    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-etc-kubernetes\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.626049    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-systemd\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.626053    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-multus-cni-dir\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.626076    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-system-cni-dir\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.626100    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-modprobe-d\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.626126    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3668c731-b9db-4e02-a8e7-16357f247006-iptables-alerter-script\") pod \"iptables-alerter-fpj7r\" (UID: \"3668c731-b9db-4e02-a8e7-16357f247006\") " pod="openshift-network-operator/iptables-alerter-fpj7r"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.626137    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-host-var-lib-cni-multus\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.626102    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-etc-kubernetes\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.626249    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/826f2d27-7719-4185-8e92-d020c83a8409-system-cni-dir\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.626280    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-modprobe-d\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.626290    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-systemd\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.626644    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3668c731-b9db-4e02-a8e7-16357f247006-iptables-alerter-script\") pod \"iptables-alerter-fpj7r\" (UID: \"3668c731-b9db-4e02-a8e7-16357f247006\") " pod="openshift-network-operator/iptables-alerter-fpj7r"
Jul 20 16:45:20.627442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.627168    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b3bec34-0628-43fe-9211-2203141bd83c-agent-certs\") pod \"konnectivity-agent-8phrk\" (UID: \"9b3bec34-0628-43fe-9211-2203141bd83c\") " pod="kube-system/konnectivity-agent-8phrk"
Jul 20 16:45:20.628059 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.627811    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-tmp\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.628059 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.627999    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-etc-tuned\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.632450 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.632174    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhrrk\" (UniqueName: \"kubernetes.io/projected/6d79d643-3794-4ea2-98a4-7aabcdda7572-kube-api-access-mhrrk\") pod \"aws-ebs-csi-driver-node-wgckr\" (UID: \"6d79d643-3794-4ea2-98a4-7aabcdda7572\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.636187 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.636142    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm49r\" (UniqueName: \"kubernetes.io/projected/826f2d27-7719-4185-8e92-d020c83a8409-kube-api-access-cm49r\") pod \"multus-brgfz\" (UID: \"826f2d27-7719-4185-8e92-d020c83a8409\") " pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.637095 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.637080    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-682pd\" (UniqueName: \"kubernetes.io/projected/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-kube-api-access-682pd\") pod \"network-metrics-daemon-m28fr\" (UID: \"87e02cfa-5d5b-4020-ac07-c4cf81f3e099\") " pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:20.639462 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.639443    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zps72\" (UniqueName: \"kubernetes.io/projected/798679f7-4546-4546-bc3d-c12693574272-kube-api-access-zps72\") pod \"node-resolver-6g5w2\" (UID: \"798679f7-4546-4546-bc3d-c12693574272\") " pod="openshift-dns/node-resolver-6g5w2"
Jul 20 16:45:20.639914 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.639891    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnjkd\" (UniqueName: \"kubernetes.io/projected/46c16140-8350-4ab4-9e33-b92a7fb16f1e-kube-api-access-gnjkd\") pod \"node-ca-sfzwq\" (UID: \"46c16140-8350-4ab4-9e33-b92a7fb16f1e\") " pod="openshift-image-registry/node-ca-sfzwq"
Jul 20 16:45:20.640017 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.639917    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r9nt\" (UniqueName: \"kubernetes.io/projected/3668c731-b9db-4e02-a8e7-16357f247006-kube-api-access-5r9nt\") pod \"iptables-alerter-fpj7r\" (UID: \"3668c731-b9db-4e02-a8e7-16357f247006\") " pod="openshift-network-operator/iptables-alerter-fpj7r"
Jul 20 16:45:20.641091 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:20.641068    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeab54f1096556ef0bdc91b7b84f5a56.slice/crio-612949fcc2ef2948dc88f1dbefaeb8afcfc808e81a3e168bbb798c4e16cf93fc WatchSource:0}: Error finding container 612949fcc2ef2948dc88f1dbefaeb8afcfc808e81a3e168bbb798c4e16cf93fc: Status 404 returned error can't find the container with id 612949fcc2ef2948dc88f1dbefaeb8afcfc808e81a3e168bbb798c4e16cf93fc
Jul 20 16:45:20.641091 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.641075    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgfgq\" (UniqueName: \"kubernetes.io/projected/ddcb791f-279c-4c1f-96a0-3b50b6c7f373-kube-api-access-cgfgq\") pod \"tuned-wbchd\" (UID: \"ddcb791f-279c-4c1f-96a0-3b50b6c7f373\") " pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.641708 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:20.641688    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1dddf4097058c612737a8bf9968fc66.slice/crio-7f2c7d4d9ccdf41a2ce8da2652ac88222b9d1d60d821686c77673e8b5d66cd22 WatchSource:0}: Error finding container 7f2c7d4d9ccdf41a2ce8da2652ac88222b9d1d60d821686c77673e8b5d66cd22: Status 404 returned error can't find the container with id 7f2c7d4d9ccdf41a2ce8da2652ac88222b9d1d60d821686c77673e8b5d66cd22
Jul 20 16:45:20.646331 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.646314    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 16:45:20.744027 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.743957    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f844c"
Jul 20 16:45:20.754011 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:20.753989    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff628bb3_d193_412f_bae5_c23349f488da.slice/crio-29fe9e8e08621f939d18f29794811ed3206d0d3d0e6176677433255e6e625f6e WatchSource:0}: Error finding container 29fe9e8e08621f939d18f29794811ed3206d0d3d0e6176677433255e6e625f6e: Status 404 returned error can't find the container with id 29fe9e8e08621f939d18f29794811ed3206d0d3d0e6176677433255e6e625f6e
Jul 20 16:45:20.755593 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.755575    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:20.763123 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:20.763105    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b664df_5e35_4825_a3ae_e8310a71b97e.slice/crio-b18dc6114c46e4d2dc6d9d870ed188aeaf59c9cfe974852e6b16383cacc2fd3c WatchSource:0}: Error finding container b18dc6114c46e4d2dc6d9d870ed188aeaf59c9cfe974852e6b16383cacc2fd3c: Status 404 returned error can't find the container with id b18dc6114c46e4d2dc6d9d870ed188aeaf59c9cfe974852e6b16383cacc2fd3c
Jul 20 16:45:20.773801 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.773782    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6g5w2"
Jul 20 16:45:20.775961 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.775947    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-brgfz"
Jul 20 16:45:20.781515 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:20.781486    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod798679f7_4546_4546_bc3d_c12693574272.slice/crio-eca362bd14ef919b463e48b0d2eec7771194149fec29c7a4c39584344639bc3a WatchSource:0}: Error finding container eca362bd14ef919b463e48b0d2eec7771194149fec29c7a4c39584344639bc3a: Status 404 returned error can't find the container with id eca362bd14ef919b463e48b0d2eec7771194149fec29c7a4c39584344639bc3a
Jul 20 16:45:20.783581 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:20.783561    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod826f2d27_7719_4185_8e92_d020c83a8409.slice/crio-ff137fc87b42c998bbc0c61323d5b1ce272ce33ffac5cc84b9261707aa7a0416 WatchSource:0}: Error finding container ff137fc87b42c998bbc0c61323d5b1ce272ce33ffac5cc84b9261707aa7a0416: Status 404 returned error can't find the container with id ff137fc87b42c998bbc0c61323d5b1ce272ce33ffac5cc84b9261707aa7a0416
Jul 20 16:45:20.798443 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.798424    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wbchd"
Jul 20 16:45:20.807732 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:20.807708    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddcb791f_279c_4c1f_96a0_3b50b6c7f373.slice/crio-f09868c90f216763a38a5cb2117fade2ba9ed08637dd55ecbe04e4058c90d571 WatchSource:0}: Error finding container f09868c90f216763a38a5cb2117fade2ba9ed08637dd55ecbe04e4058c90d571: Status 404 returned error can't find the container with id f09868c90f216763a38a5cb2117fade2ba9ed08637dd55ecbe04e4058c90d571
Jul 20 16:45:20.817300 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.817283    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fpj7r"
Jul 20 16:45:20.825012 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:20.824993    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3668c731_b9db_4e02_a8e7_16357f247006.slice/crio-3d661e31a460c19b50064637b42310dc00b2ba010ff3e494b57d93ccd6ace4fe WatchSource:0}: Error finding container 3d661e31a460c19b50064637b42310dc00b2ba010ff3e494b57d93ccd6ace4fe: Status 404 returned error can't find the container with id 3d661e31a460c19b50064637b42310dc00b2ba010ff3e494b57d93ccd6ace4fe
Jul 20 16:45:20.832680 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.832662    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr"
Jul 20 16:45:20.840301 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:20.840281    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d79d643_3794_4ea2_98a4_7aabcdda7572.slice/crio-d9bbbd952795328de35b752d6c44bfaa2c11673f41613352adfd026783dba992 WatchSource:0}: Error finding container d9bbbd952795328de35b752d6c44bfaa2c11673f41613352adfd026783dba992: Status 404 returned error can't find the container with id d9bbbd952795328de35b752d6c44bfaa2c11673f41613352adfd026783dba992
Jul 20 16:45:20.854819 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.854801    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8phrk"
Jul 20 16:45:20.859385 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:20.859372    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sfzwq"
Jul 20 16:45:20.864887 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:20.864861    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b3bec34_0628_43fe_9211_2203141bd83c.slice/crio-a49ecca75d5626677767c31088ce8d20cd492c7b5a2d66a8448647f651519515 WatchSource:0}: Error finding container a49ecca75d5626677767c31088ce8d20cd492c7b5a2d66a8448647f651519515: Status 404 returned error can't find the container with id a49ecca75d5626677767c31088ce8d20cd492c7b5a2d66a8448647f651519515
Jul 20 16:45:20.868055 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:20.868039    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c16140_8350_4ab4_9e33_b92a7fb16f1e.slice/crio-55677b9e361d45f9253557e0022d59effe3f5a389b2930dee0dc0ba167925516 WatchSource:0}: Error finding container 55677b9e361d45f9253557e0022d59effe3f5a389b2930dee0dc0ba167925516: Status 404 returned error can't find the container with id 55677b9e361d45f9253557e0022d59effe3f5a389b2930dee0dc0ba167925516
Jul 20 16:45:22.260926 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.260898    2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 16:45:22.379787 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.379714    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtcj7\" (UniqueName: \"kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7\") pod \"network-check-target-ql4vb\" (UID: \"ef355774-d5d7-4b50-9643-9ba08afd080d\") " pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:22.379920 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:22.379830    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:45:22.379920 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:22.379845    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:45:22.379920 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:22.379854    2568 projected.go:196] Error preparing data for projected volume kube-api-access-qtcj7 for pod openshift-network-diagnostics/network-check-target-ql4vb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:22.379920 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:22.379896    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7 podName:ef355774-d5d7-4b50-9643-9ba08afd080d nodeName:}" failed. No retries permitted until 2026-07-20 16:45:23.379884037 +0000 UTC m=+2.927740081 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qtcj7" (UniqueName: "kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7") pod "network-check-target-ql4vb" (UID: "ef355774-d5d7-4b50-9643-9ba08afd080d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:22.480172 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.480134    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs\") pod \"network-metrics-daemon-m28fr\" (UID: \"87e02cfa-5d5b-4020-ac07-c4cf81f3e099\") " pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:22.480172 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.480176    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:22.480383 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:22.480286    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:22.480383 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:22.480288    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:22.480383 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:22.480336    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret podName:e1e4a352-901a-4050-b0c4-5e63ec823420 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:23.480324329 +0000 UTC m=+3.028180373 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret") pod "global-pull-secret-syncer-ntr6q" (UID: "e1e4a352-901a-4050-b0c4-5e63ec823420") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:22.480383 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:22.480351    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs podName:87e02cfa-5d5b-4020-ac07-c4cf81f3e099 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:23.480344094 +0000 UTC m=+3.028200138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs") pod "network-metrics-daemon-m28fr" (UID: "87e02cfa-5d5b-4020-ac07-c4cf81f3e099") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:22.830673 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.830560    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 16:40:21 +0000 UTC" deadline="2028-01-11 07:11:17.830788881 +0000 UTC"
Jul 20 16:45:22.830673 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.830632    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12950h25m55.00016074s"
Jul 20 16:45:22.890709 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.890682    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:22.890865 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:22.890810    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:22.891216 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.891194    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:22.891365 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:22.891323    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:22.891523 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.891508    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:22.891625 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:22.891607    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:22.901961 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.901916    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wbchd" event={"ID":"ddcb791f-279c-4c1f-96a0-3b50b6c7f373","Type":"ContainerStarted","Data":"f09868c90f216763a38a5cb2117fade2ba9ed08637dd55ecbe04e4058c90d571"}
Jul 20 16:45:22.905757 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.905730    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6g5w2" event={"ID":"798679f7-4546-4546-bc3d-c12693574272","Type":"ContainerStarted","Data":"eca362bd14ef919b463e48b0d2eec7771194149fec29c7a4c39584344639bc3a"}
Jul 20 16:45:22.912615 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.912508    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk" event={"ID":"75b664df-5e35-4825-a3ae-e8310a71b97e","Type":"ContainerStarted","Data":"b18dc6114c46e4d2dc6d9d870ed188aeaf59c9cfe974852e6b16383cacc2fd3c"}
Jul 20 16:45:22.923622 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.923565    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f844c" event={"ID":"ff628bb3-d193-412f-bae5-c23349f488da","Type":"ContainerStarted","Data":"29fe9e8e08621f939d18f29794811ed3206d0d3d0e6176677433255e6e625f6e"}
Jul 20 16:45:22.930040 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.929960    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal" event={"ID":"feab54f1096556ef0bdc91b7b84f5a56","Type":"ContainerStarted","Data":"612949fcc2ef2948dc88f1dbefaeb8afcfc808e81a3e168bbb798c4e16cf93fc"}
Jul 20 16:45:22.931711 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.931662    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-8.ec2.internal" event={"ID":"e1dddf4097058c612737a8bf9968fc66","Type":"ContainerStarted","Data":"7f2c7d4d9ccdf41a2ce8da2652ac88222b9d1d60d821686c77673e8b5d66cd22"}
Jul 20 16:45:22.935301 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.935222    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sfzwq" event={"ID":"46c16140-8350-4ab4-9e33-b92a7fb16f1e","Type":"ContainerStarted","Data":"55677b9e361d45f9253557e0022d59effe3f5a389b2930dee0dc0ba167925516"}
Jul 20 16:45:22.957695 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.957566    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr" event={"ID":"6d79d643-3794-4ea2-98a4-7aabcdda7572","Type":"ContainerStarted","Data":"d9bbbd952795328de35b752d6c44bfaa2c11673f41613352adfd026783dba992"}
Jul 20 16:45:22.985574 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.985388    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fpj7r" event={"ID":"3668c731-b9db-4e02-a8e7-16357f247006","Type":"ContainerStarted","Data":"3d661e31a460c19b50064637b42310dc00b2ba010ff3e494b57d93ccd6ace4fe"}
Jul 20 16:45:22.993883 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:22.993783    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-brgfz" event={"ID":"826f2d27-7719-4185-8e92-d020c83a8409","Type":"ContainerStarted","Data":"ff137fc87b42c998bbc0c61323d5b1ce272ce33ffac5cc84b9261707aa7a0416"}
Jul 20 16:45:23.013465 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:23.013408    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8phrk" event={"ID":"9b3bec34-0628-43fe-9211-2203141bd83c","Type":"ContainerStarted","Data":"a49ecca75d5626677767c31088ce8d20cd492c7b5a2d66a8448647f651519515"}
Jul 20 16:45:23.293013 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:23.292765    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 16:45:23.388989 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:23.388952    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtcj7\" (UniqueName: \"kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7\") pod \"network-check-target-ql4vb\" (UID: \"ef355774-d5d7-4b50-9643-9ba08afd080d\") " pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:23.389178 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:23.389144    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:45:23.389178 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:23.389163    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:45:23.389178 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:23.389178    2568 projected.go:196] Error preparing data for projected volume kube-api-access-qtcj7 for pod openshift-network-diagnostics/network-check-target-ql4vb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:23.389370 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:23.389253    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7 podName:ef355774-d5d7-4b50-9643-9ba08afd080d nodeName:}" failed. No retries permitted until 2026-07-20 16:45:25.389216755 +0000 UTC m=+4.937072805 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qtcj7" (UniqueName: "kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7") pod "network-check-target-ql4vb" (UID: "ef355774-d5d7-4b50-9643-9ba08afd080d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:23.489527 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:23.489490    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs\") pod \"network-metrics-daemon-m28fr\" (UID: \"87e02cfa-5d5b-4020-ac07-c4cf81f3e099\") " pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:23.489693 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:23.489557    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:23.489752 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:23.489704    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:23.489831 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:23.489796    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret podName:e1e4a352-901a-4050-b0c4-5e63ec823420 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:25.489744892 +0000 UTC m=+5.037600943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret") pod "global-pull-secret-syncer-ntr6q" (UID: "e1e4a352-901a-4050-b0c4-5e63ec823420") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:23.490331 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:23.490312    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:23.490448 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:23.490364    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs podName:87e02cfa-5d5b-4020-ac07-c4cf81f3e099 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:25.49034911 +0000 UTC m=+5.038205159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs") pod "network-metrics-daemon-m28fr" (UID: "87e02cfa-5d5b-4020-ac07-c4cf81f3e099") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:23.710437 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:23.710358    2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 16:45:23.830955 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:23.830918    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 16:40:21 +0000 UTC" deadline="2027-12-14 17:19:50.93577611 +0000 UTC"
Jul 20 16:45:23.830955 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:23.830954    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12288h34m27.104827077s"
Jul 20 16:45:24.891297 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:24.890637    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:24.891297 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:24.890765    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:24.891297 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:24.891127    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:24.891297 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:24.891229    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:24.892172 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:24.892004    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:24.892172 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:24.892102    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:25.407138 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:25.407091    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtcj7\" (UniqueName: \"kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7\") pod \"network-check-target-ql4vb\" (UID: \"ef355774-d5d7-4b50-9643-9ba08afd080d\") " pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:25.407354 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:25.407334    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:45:25.407438 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:25.407361    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:45:25.407438 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:25.407375    2568 projected.go:196] Error preparing data for projected volume kube-api-access-qtcj7 for pod openshift-network-diagnostics/network-check-target-ql4vb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:25.407542 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:25.407439    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7 podName:ef355774-d5d7-4b50-9643-9ba08afd080d nodeName:}" failed. No retries permitted until 2026-07-20 16:45:29.407420197 +0000 UTC m=+8.955276256 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qtcj7" (UniqueName: "kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7") pod "network-check-target-ql4vb" (UID: "ef355774-d5d7-4b50-9643-9ba08afd080d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:25.508383 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:25.508344    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs\") pod \"network-metrics-daemon-m28fr\" (UID: \"87e02cfa-5d5b-4020-ac07-c4cf81f3e099\") " pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:25.508558 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:25.508413    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:25.508558 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:25.508539    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:25.508670 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:25.508573    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:25.508670 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:25.508639    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs podName:87e02cfa-5d5b-4020-ac07-c4cf81f3e099 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:29.508604936 +0000 UTC m=+9.056460999 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs") pod "network-metrics-daemon-m28fr" (UID: "87e02cfa-5d5b-4020-ac07-c4cf81f3e099") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:25.508670 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:25.508663    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret podName:e1e4a352-901a-4050-b0c4-5e63ec823420 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:29.508651382 +0000 UTC m=+9.056507430 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret") pod "global-pull-secret-syncer-ntr6q" (UID: "e1e4a352-901a-4050-b0c4-5e63ec823420") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:26.892858 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:26.892194    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:26.892858 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:26.892323    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:26.892858 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:26.892629    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:26.892858 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:26.892698    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:26.892858 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:26.892737    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:26.892858 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:26.892780    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:28.893004 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:28.892216    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:28.893004 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:28.892359    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:28.893004 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:28.892726    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:28.893004 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:28.892830    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:28.893004 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:28.892879    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:28.893004 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:28.892952    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:29.441617 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:29.440910    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtcj7\" (UniqueName: \"kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7\") pod \"network-check-target-ql4vb\" (UID: \"ef355774-d5d7-4b50-9643-9ba08afd080d\") " pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:29.441617 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:29.441202    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:45:29.441617 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:29.441224    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:45:29.441617 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:29.441253    2568 projected.go:196] Error preparing data for projected volume kube-api-access-qtcj7 for pod openshift-network-diagnostics/network-check-target-ql4vb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:29.441617 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:29.441310    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7 podName:ef355774-d5d7-4b50-9643-9ba08afd080d nodeName:}" failed. No retries permitted until 2026-07-20 16:45:37.44129204 +0000 UTC m=+16.989148086 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qtcj7" (UniqueName: "kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7") pod "network-check-target-ql4vb" (UID: "ef355774-d5d7-4b50-9643-9ba08afd080d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:29.541687 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:29.541650    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs\") pod \"network-metrics-daemon-m28fr\" (UID: \"87e02cfa-5d5b-4020-ac07-c4cf81f3e099\") " pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:29.541871 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:29.541715    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:29.541926 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:29.541878    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:29.541976 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:29.541936    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret podName:e1e4a352-901a-4050-b0c4-5e63ec823420 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:37.541922749 +0000 UTC m=+17.089778796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret") pod "global-pull-secret-syncer-ntr6q" (UID: "e1e4a352-901a-4050-b0c4-5e63ec823420") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:29.542272 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:29.542251    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:29.542399 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:29.542289    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs podName:87e02cfa-5d5b-4020-ac07-c4cf81f3e099 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:37.542276739 +0000 UTC m=+17.090132782 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs") pod "network-metrics-daemon-m28fr" (UID: "87e02cfa-5d5b-4020-ac07-c4cf81f3e099") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:30.890790 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:30.890762    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:30.891257 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:30.890839    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:30.891257 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:30.890942    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:30.891257 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:30.891066    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:30.891257 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:30.891087    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:30.891257 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:30.891141    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:32.890301 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:32.890089    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:32.890729 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:32.890113    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:32.890729 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:32.890399    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:32.890729 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:32.890147    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:32.890729 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:32.890453    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:32.890729 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:32.890599    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:34.890130 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:34.890094    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:34.890130 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:34.890119    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:34.890617 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:34.890119    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:34.890617 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:34.890214    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:34.890617 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:34.890339    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:34.890617 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:34.890440    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:36.890515 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:36.890052    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:36.890515 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:36.890079    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:36.890515 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:36.890104    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:36.890515 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:36.890178    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:36.890515 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:36.890290    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:36.890515 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:36.890404    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:37.499461 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:37.499429    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtcj7\" (UniqueName: \"kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7\") pod \"network-check-target-ql4vb\" (UID: \"ef355774-d5d7-4b50-9643-9ba08afd080d\") " pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:37.499625 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:37.499553    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:45:37.499625 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:37.499576    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:45:37.499625 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:37.499585    2568 projected.go:196] Error preparing data for projected volume kube-api-access-qtcj7 for pod openshift-network-diagnostics/network-check-target-ql4vb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:37.499766 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:37.499636    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7 podName:ef355774-d5d7-4b50-9643-9ba08afd080d nodeName:}" failed. No retries permitted until 2026-07-20 16:45:53.499620771 +0000 UTC m=+33.047476814 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qtcj7" (UniqueName: "kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7") pod "network-check-target-ql4vb" (UID: "ef355774-d5d7-4b50-9643-9ba08afd080d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:37.600383 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:37.600344    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs\") pod \"network-metrics-daemon-m28fr\" (UID: \"87e02cfa-5d5b-4020-ac07-c4cf81f3e099\") " pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:37.600567 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:37.600403    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:37.600567 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:37.600506    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:37.600567 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:37.600533    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:37.600567 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:37.600568    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs podName:87e02cfa-5d5b-4020-ac07-c4cf81f3e099 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:53.600550317 +0000 UTC m=+33.148406378 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs") pod "network-metrics-daemon-m28fr" (UID: "87e02cfa-5d5b-4020-ac07-c4cf81f3e099") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:37.600752 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:37.600584    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret podName:e1e4a352-901a-4050-b0c4-5e63ec823420 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:53.60057746 +0000 UTC m=+33.148433504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret") pod "global-pull-secret-syncer-ntr6q" (UID: "e1e4a352-901a-4050-b0c4-5e63ec823420") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:38.893544 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:38.893108    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:38.893544 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:38.893210    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:38.893544 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:38.893212    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:38.893544 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:38.893273    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:38.893544 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:38.893376    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:38.893544 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:38.893465    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:40.892910 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:40.892883    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:40.893391 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:40.892977    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:40.893391 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:40.892984    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:40.893391 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:40.893121    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:40.893391 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:40.893166    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:40.893391 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:40.893259    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:42.893763 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:42.893506    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:42.893763 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:42.893724    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:42.893763 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:42.893729    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:42.894450 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:42.893798    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:42.894450 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:42.893911    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:42.894450 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:42.894006    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:43.060688 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.060657    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-brgfz" event={"ID":"826f2d27-7719-4185-8e92-d020c83a8409","Type":"ContainerStarted","Data":"7d5300ac259745d39d4302173a13aac798e23b3bb0f517d7bbd3351e84d4494c"}
Jul 20 16:45:43.062327 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.062203    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8phrk" event={"ID":"9b3bec34-0628-43fe-9211-2203141bd83c","Type":"ContainerStarted","Data":"50342af0c77a235fa9b7962df26b44989c73f84a800a947157c46af2b9287830"}
Jul 20 16:45:43.064020 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.063778    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wbchd" event={"ID":"ddcb791f-279c-4c1f-96a0-3b50b6c7f373","Type":"ContainerStarted","Data":"ca03c79e366940dca66003aff1905cfc2fd3db65d9df093aae167073fa57b3c6"}
Jul 20 16:45:43.065482 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.065462    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6g5w2" event={"ID":"798679f7-4546-4546-bc3d-c12693574272","Type":"ContainerStarted","Data":"b631d1e90832f1be7619b14c059bbeff9ff6517fcfefeb62b29fdcdd206c2ff5"}
Jul 20 16:45:43.068334 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.068317    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 16:45:43.068775 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.068748    2568 generic.go:356] "Generic (PLEG): container finished" podID="75b664df-5e35-4825-a3ae-e8310a71b97e" containerID="74e6f5cf924473ac8c1fdc3c936a2af8991b4ecf24d230e6ba627d16d7fb9076" exitCode=1
Jul 20 16:45:43.068851 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.068784    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk" event={"ID":"75b664df-5e35-4825-a3ae-e8310a71b97e","Type":"ContainerStarted","Data":"0cd9eba668b0eae861736098d8a8d2dda8a270386c478ed0f2951946ca6f39aa"}
Jul 20 16:45:43.068851 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.068839    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk" event={"ID":"75b664df-5e35-4825-a3ae-e8310a71b97e","Type":"ContainerStarted","Data":"f50764faf92e5040b4c1c3df49e3eab879c7951b38d8a544814ff58d8e87dbb1"}
Jul 20 16:45:43.068930 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.068876    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk" event={"ID":"75b664df-5e35-4825-a3ae-e8310a71b97e","Type":"ContainerStarted","Data":"25e7c61f78d9885ae16b94c4c3c65767201007b868caa0272fbf65d96183fbfe"}
Jul 20 16:45:43.068930 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.068917    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk" event={"ID":"75b664df-5e35-4825-a3ae-e8310a71b97e","Type":"ContainerStarted","Data":"e8f82213d1cbd2e351a58d4f4316a200ca5e78d56e4d181627e36619c5ec1935"}
Jul 20 16:45:43.068985 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.068969    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk" event={"ID":"75b664df-5e35-4825-a3ae-e8310a71b97e","Type":"ContainerDied","Data":"74e6f5cf924473ac8c1fdc3c936a2af8991b4ecf24d230e6ba627d16d7fb9076"}
Jul 20 16:45:43.069021 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.069001    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk" event={"ID":"75b664df-5e35-4825-a3ae-e8310a71b97e","Type":"ContainerStarted","Data":"7d135d14f633f7a0ab2775fab28a4925615ae854eb9e34e8a4fd8aa390a3c0cc"}
Jul 20 16:45:43.070412 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.070392    2568 generic.go:356] "Generic (PLEG): container finished" podID="ff628bb3-d193-412f-bae5-c23349f488da" containerID="747d6c2e8802ed318c18e512b9ffa4afc0424c8d22ae265cd80aebcd785c7d24" exitCode=0
Jul 20 16:45:43.070509 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.070465    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f844c" event={"ID":"ff628bb3-d193-412f-bae5-c23349f488da","Type":"ContainerDied","Data":"747d6c2e8802ed318c18e512b9ffa4afc0424c8d22ae265cd80aebcd785c7d24"}
Jul 20 16:45:43.072210 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.072188    2568 generic.go:356] "Generic (PLEG): container finished" podID="feab54f1096556ef0bdc91b7b84f5a56" containerID="55af63f48d88780326fd316ded37bffd2bac160e6292099e9658b39e0a9c958e" exitCode=0
Jul 20 16:45:43.075271 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.072430    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal" event={"ID":"feab54f1096556ef0bdc91b7b84f5a56","Type":"ContainerDied","Data":"55af63f48d88780326fd316ded37bffd2bac160e6292099e9658b39e0a9c958e"}
Jul 20 16:45:43.077858 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.077832    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-8.ec2.internal" event={"ID":"e1dddf4097058c612737a8bf9968fc66","Type":"ContainerStarted","Data":"e60fbb1ac3eb36505ec24fa5e3577e9c2017599e498639a654dc5280e602b54e"}
Jul 20 16:45:43.079213 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.078178    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-brgfz" podStartSLOduration=2.650038269 podStartE2EDuration="23.078144585s" totalImagesPullingTime="20.428106316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:43.078126113 +0000 UTC m=+22.625982181" watchObservedRunningTime="2026-07-20 16:45:43.078144585 +0000 UTC m=+22.626000645"
Jul 20 16:45:43.079504 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.079478    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sfzwq" event={"ID":"46c16140-8350-4ab4-9e33-b92a7fb16f1e","Type":"ContainerStarted","Data":"6cb967f7284745786f560f99bab844d9a7b32581bcbe7a61da20147d9f441048"}
Jul 20 16:45:43.080857 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.080835    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr" event={"ID":"6d79d643-3794-4ea2-98a4-7aabcdda7572","Type":"ContainerStarted","Data":"dd5af8322d49c6b70f093a0c8b1cdba72c2cc582e223c2993d470d49176cbba1"}
Jul 20 16:45:43.110812 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.110774    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6g5w2" podStartSLOduration=3.019975389 podStartE2EDuration="23.110760342s" totalImagesPullingTime="20.090784953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:43.11063671 +0000 UTC m=+22.658492775" watchObservedRunningTime="2026-07-20 16:45:43.110760342 +0000 UTC m=+22.658616407"
Jul 20 16:45:43.130200 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.129996    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wbchd" podStartSLOduration=2.062981973 podStartE2EDuration="22.12998401s" totalImagesPullingTime="20.067002037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:43.129855063 +0000 UTC m=+22.677711129" watchObservedRunningTime="2026-07-20 16:45:43.12998401 +0000 UTC m=+22.677840074"
Jul 20 16:45:43.145770 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.145726    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8phrk" podStartSLOduration=2.13744411 podStartE2EDuration="22.145712074s" totalImagesPullingTime="20.008267964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:43.145141938 +0000 UTC m=+22.692998003" watchObservedRunningTime="2026-07-20 16:45:43.145712074 +0000 UTC m=+22.693568138"
Jul 20 16:45:43.175805 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.175765    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-8.ec2.internal" podStartSLOduration=22.175744292 podStartE2EDuration="22.175744292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:43.175440419 +0000 UTC m=+22.723296485" watchObservedRunningTime="2026-07-20 16:45:43.175744292 +0000 UTC m=+22.723600357"
Jul 20 16:45:43.481231 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.481138    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8phrk"
Jul 20 16:45:43.482071 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.482058    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8phrk"
Jul 20 16:45:43.497641 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:43.497602    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sfzwq" podStartSLOduration=2.492190069 podStartE2EDuration="22.497588349s" totalImagesPullingTime="20.00539828s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:43.196154666 +0000 UTC m=+22.744010732" watchObservedRunningTime="2026-07-20 16:45:43.497588349 +0000 UTC m=+23.045444413"
Jul 20 16:45:44.084361 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:44.084277    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal" event={"ID":"feab54f1096556ef0bdc91b7b84f5a56","Type":"ContainerStarted","Data":"11a60b872e84a8ba3332707d6f26a2c365a20e3d0ce9ebb0be6d398768d3f90b"}
Jul 20 16:45:44.085685 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:44.085663    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fpj7r" event={"ID":"3668c731-b9db-4e02-a8e7-16357f247006","Type":"ContainerStarted","Data":"b7d7a47352da7d49d25d1fce97c1d387cd694db5032f937bf7e0d87fa456d3c9"}
Jul 20 16:45:44.086320 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:44.086301    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8phrk"
Jul 20 16:45:44.086413 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:44.086350    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8phrk"
Jul 20 16:45:44.098969 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:44.098936    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-8.ec2.internal" podStartSLOduration=23.098923885 podStartE2EDuration="23.098923885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:44.098753885 +0000 UTC m=+23.646609949" watchObservedRunningTime="2026-07-20 16:45:44.098923885 +0000 UTC m=+23.646779950"
Jul 20 16:45:44.126751 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:44.126712    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fpj7r" podStartSLOduration=3.076808836 podStartE2EDuration="23.126701177s" totalImagesPullingTime="20.049892341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:44.126507542 +0000 UTC m=+23.674363608" watchObservedRunningTime="2026-07-20 16:45:44.126701177 +0000 UTC m=+23.674557241"
Jul 20 16:45:44.473201 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:44.473173    2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 20 16:45:44.841000 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:44.840829    2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T16:45:44.473197319Z","UUID":"111498e7-dd3d-414e-a05c-2c266980773c","Handler":null,"Name":"","Endpoint":""}
Jul 20 16:45:44.842521 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:44.842503    2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 20 16:45:44.842625 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:44.842537    2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 20 16:45:44.890617 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:44.890573    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:44.890617 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:44.890591    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:44.890818 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:44.890720    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:44.890818 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:44.890776    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:44.890917 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:44.890895    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:44.891032 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:44.891003    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:45.089523 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:45.089487    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr" event={"ID":"6d79d643-3794-4ea2-98a4-7aabcdda7572","Type":"ContainerStarted","Data":"b23ed2f98ceca829451df61ce8d794a1c47da247ff3419074c8238a476307a8c"}
Jul 20 16:45:46.094909 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:46.094880    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 16:45:46.095412 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:46.095266    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk" event={"ID":"75b664df-5e35-4825-a3ae-e8310a71b97e","Type":"ContainerStarted","Data":"347f471841e5729362804358da470f316c072c169b87e453230f194e6efdb2ad"}
Jul 20 16:45:46.890624 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:46.890591    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:46.890789 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:46.890633    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:46.890789 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:46.890591    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:46.890789 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:46.890707    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:46.890908 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:46.890810    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:46.890908 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:46.890888    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:47.037396 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:47.037330    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:47.100269 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:47.100224    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr" event={"ID":"6d79d643-3794-4ea2-98a4-7aabcdda7572","Type":"ContainerStarted","Data":"65b9d46646bbd1c39ebf4c70360ffb541804d40aad9e86b89f410efbe6d54429"}
Jul 20 16:45:47.119682 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:47.119632    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wgckr" podStartSLOduration=3.023138873 podStartE2EDuration="26.119612853s" totalImagesPullingTime="23.09647398s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:47.118749051 +0000 UTC m=+26.666605127" watchObservedRunningTime="2026-07-20 16:45:47.119612853 +0000 UTC m=+26.667468919"
Jul 20 16:45:47.122690 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:47.122592    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:48.892903 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:48.892744    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:48.893589 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:48.892797    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:48.893589 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:48.892981    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:48.893589 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:48.893036    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:48.893589 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:48.892797    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:48.893589 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:48.893097    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:49.107355 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:49.107330    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 16:45:49.107673 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:49.107652    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk" event={"ID":"75b664df-5e35-4825-a3ae-e8310a71b97e","Type":"ContainerStarted","Data":"c6bae1380ea1d262b22d4fbddbe2c0d610db4382dda9e36f56f592b86c1a72cb"}
Jul 20 16:45:49.107976 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:49.107942    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:49.108192 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:49.108177    2568 scope.go:117] "RemoveContainer" containerID="74e6f5cf924473ac8c1fdc3c936a2af8991b4ecf24d230e6ba627d16d7fb9076"
Jul 20 16:45:49.109807 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:49.109785    2568 generic.go:356] "Generic (PLEG): container finished" podID="ff628bb3-d193-412f-bae5-c23349f488da" containerID="92b52da1b9fbebe3040be113f383bfdbab516ef3ff8354f7ab40c0f8a7082731" exitCode=0
Jul 20 16:45:49.109901 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:49.109826    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f844c" event={"ID":"ff628bb3-d193-412f-bae5-c23349f488da","Type":"ContainerDied","Data":"92b52da1b9fbebe3040be113f383bfdbab516ef3ff8354f7ab40c0f8a7082731"}
Jul 20 16:45:49.125783 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:49.125763    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:49.659884 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:49.659697    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:49.718757 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:49.718725    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:50.117186 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:50.117164    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 16:45:50.117620 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:50.117589    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk" event={"ID":"75b664df-5e35-4825-a3ae-e8310a71b97e","Type":"ContainerStarted","Data":"1e9d51722877fcfb03988930f6cbdb8c7cb34165bfddfe7a32272db4652fa02e"}
Jul 20 16:45:50.118554 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:50.118537    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:50.118621 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:50.118598    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:50.134509 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:50.134444    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:45:50.145664 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:50.145630    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk" podStartSLOduration=10.009884281 podStartE2EDuration="30.145616807s" totalImagesPullingTime="20.135732526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:50.144159578 +0000 UTC m=+29.692015643" watchObservedRunningTime="2026-07-20 16:45:50.145616807 +0000 UTC m=+29.693472871"
Jul 20 16:45:50.463610 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:50.463576    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m28fr"]
Jul 20 16:45:50.463788 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:50.463733    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:50.463850 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:50.463834    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:50.466978 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:50.466941    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ntr6q"]
Jul 20 16:45:50.467081 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:50.467047    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:50.467150 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:50.467134    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:50.467657 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:50.467635    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ql4vb"]
Jul 20 16:45:50.467770 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:50.467726    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:50.467826 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:50.467812    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:51.123667 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:51.123630    2568 generic.go:356] "Generic (PLEG): container finished" podID="ff628bb3-d193-412f-bae5-c23349f488da" containerID="92bd9e761e2091934fa54b1b3fb9263dc19e04f739843d960b087c34bf5ea699" exitCode=0
Jul 20 16:45:51.124197 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:51.123720    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f844c" event={"ID":"ff628bb3-d193-412f-bae5-c23349f488da","Type":"ContainerDied","Data":"92bd9e761e2091934fa54b1b3fb9263dc19e04f739843d960b087c34bf5ea699"}
Jul 20 16:45:51.650786 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:51.650757    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:51.870192 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:51.870156    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:51.890612 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:51.890588    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:51.890733 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:51.890691    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:51.890775 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:51.890743    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:51.890861 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:51.890846    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:52.036511 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:52.036482    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:52.893749 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:52.893712    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:52.894189 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:52.893827    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:53.130170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:53.130135    2568 generic.go:356] "Generic (PLEG): container finished" podID="ff628bb3-d193-412f-bae5-c23349f488da" containerID="90d422549f88e11a983ca75cf8d5ae39762c8e5243e01c6d0dc6c0b9340f27e4" exitCode=0
Jul 20 16:45:53.130329 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:53.130188    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f844c" event={"ID":"ff628bb3-d193-412f-bae5-c23349f488da","Type":"ContainerDied","Data":"90d422549f88e11a983ca75cf8d5ae39762c8e5243e01c6d0dc6c0b9340f27e4"}
Jul 20 16:45:53.528082 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:53.527851    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtcj7\" (UniqueName: \"kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7\") pod \"network-check-target-ql4vb\" (UID: \"ef355774-d5d7-4b50-9643-9ba08afd080d\") " pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:53.528285 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:53.528032    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:45:53.528285 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:53.528154    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:45:53.528285 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:53.528167    2568 projected.go:196] Error preparing data for projected volume kube-api-access-qtcj7 for pod openshift-network-diagnostics/network-check-target-ql4vb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:53.528285 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:53.528222    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7 podName:ef355774-d5d7-4b50-9643-9ba08afd080d nodeName:}" failed. No retries permitted until 2026-07-20 16:46:25.528207514 +0000 UTC m=+65.076063579 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qtcj7" (UniqueName: "kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7") pod "network-check-target-ql4vb" (UID: "ef355774-d5d7-4b50-9643-9ba08afd080d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:45:53.628555 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:53.628525    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs\") pod \"network-metrics-daemon-m28fr\" (UID: \"87e02cfa-5d5b-4020-ac07-c4cf81f3e099\") " pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:53.628689 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:53.628564    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:53.628689 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:53.628655    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:53.628689 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:53.628660    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:53.628783 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:53.628706    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret podName:e1e4a352-901a-4050-b0c4-5e63ec823420 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:25.628690188 +0000 UTC m=+65.176546246 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret") pod "global-pull-secret-syncer-ntr6q" (UID: "e1e4a352-901a-4050-b0c4-5e63ec823420") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:45:53.628783 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:53.628721    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs podName:87e02cfa-5d5b-4020-ac07-c4cf81f3e099 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:25.628714434 +0000 UTC m=+65.176570479 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs") pod "network-metrics-daemon-m28fr" (UID: "87e02cfa-5d5b-4020-ac07-c4cf81f3e099") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:45:53.890610 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:53.890585    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:53.890762 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:53.890686    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:53.890762 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:53.890711    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:53.890834 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:53.890764    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:54.890544 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:54.890511    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:54.890993 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:54.890640    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql4vb" podUID="ef355774-d5d7-4b50-9643-9ba08afd080d"
Jul 20 16:45:55.890737 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:55.890704    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:55.891125 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:55.890707    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:55.891125 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:55.890841    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntr6q" podUID="e1e4a352-901a-4050-b0c4-5e63ec823420"
Jul 20 16:45:55.891125 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:55.890909    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m28fr" podUID="87e02cfa-5d5b-4020-ac07-c4cf81f3e099"
Jul 20 16:45:56.274382 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.274295    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-8.ec2.internal" event="NodeReady"
Jul 20 16:45:56.274536 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.274435    2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 20 16:45:56.305952 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.305914    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-557b8b5bcc-9tqwn"]
Jul 20 16:45:56.308572 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.308548    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.311008 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.310986    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 20 16:45:56.311181 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.311162    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2mkz5\""
Jul 20 16:45:56.311283 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.311216    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 20 16:45:56.311435 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.311414    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 20 16:45:56.318698 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.318673    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 20 16:45:56.318792 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.318682    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p2vpf"]
Jul 20 16:45:56.321600 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.321584    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-m4g87"]
Jul 20 16:45:56.321728 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.321707    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p2vpf"
Jul 20 16:45:56.324205 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.324182    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 20 16:45:56.324559 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.324540    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 20 16:45:56.324659 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.324636    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-47jm7\""
Jul 20 16:45:56.324904 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.324777    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 20 16:45:56.325957 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.325934    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-557b8b5bcc-9tqwn"]
Jul 20 16:45:56.326081 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.326065    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:56.328927 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.328604    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 20 16:45:56.329343 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.329324    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vsgmd\""
Jul 20 16:45:56.330827 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.329841    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 20 16:45:56.332081 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.331806    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m4g87"]
Jul 20 16:45:56.342511 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.342489    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p2vpf"]
Jul 20 16:45:56.447907 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.447868    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:56.447907 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.447906    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.448162 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.447924    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/372d96f7-664c-4583-a4f2-5f4af55157a1-config-volume\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:56.448162 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.447949    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4085a903-e1c6-4f41-9e07-66ea43060692-image-registry-private-configuration\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.448162 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.447979    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4085a903-e1c6-4f41-9e07-66ea43060692-trusted-ca\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.448162 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.448001    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4085a903-e1c6-4f41-9e07-66ea43060692-registry-certificates\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.448162 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.448073    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/372d96f7-664c-4583-a4f2-5f4af55157a1-tmp-dir\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:56.448162 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.448154    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89b7q\" (UniqueName: \"kubernetes.io/projected/d5e37e15-6c9c-4b17-b026-5ae24cbe3e04-kube-api-access-89b7q\") pod \"ingress-canary-p2vpf\" (UID: \"d5e37e15-6c9c-4b17-b026-5ae24cbe3e04\") " pod="openshift-ingress-canary/ingress-canary-p2vpf"
Jul 20 16:45:56.448401 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.448172    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr2n2\" (UniqueName: \"kubernetes.io/projected/372d96f7-664c-4583-a4f2-5f4af55157a1-kube-api-access-pr2n2\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:56.448401 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.448188    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmr9m\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-kube-api-access-jmr9m\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.448401 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.448214    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5e37e15-6c9c-4b17-b026-5ae24cbe3e04-cert\") pod \"ingress-canary-p2vpf\" (UID: \"d5e37e15-6c9c-4b17-b026-5ae24cbe3e04\") " pod="openshift-ingress-canary/ingress-canary-p2vpf"
Jul 20 16:45:56.448401 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.448230    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4085a903-e1c6-4f41-9e07-66ea43060692-ca-trust-extracted\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.448401 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.448272    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-bound-sa-token\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.448401 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.448292    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4085a903-e1c6-4f41-9e07-66ea43060692-installation-pull-secrets\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.451040 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.451018    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:56.549694 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.549608    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-bound-sa-token\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.549694 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.549659    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4085a903-e1c6-4f41-9e07-66ea43060692-installation-pull-secrets\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.549895 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.549703    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:56.549895 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.549727    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.549895 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.549755    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/372d96f7-664c-4583-a4f2-5f4af55157a1-config-volume\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:56.550049 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:56.550024    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 16:45:56.550170 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:56.550108    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls podName:372d96f7-664c-4583-a4f2-5f4af55157a1 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:57.050085665 +0000 UTC m=+36.597941714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls") pod "dns-default-m4g87" (UID: "372d96f7-664c-4583-a4f2-5f4af55157a1") : secret "dns-default-metrics-tls" not found
Jul 20 16:45:56.550817 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.550351    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/372d96f7-664c-4583-a4f2-5f4af55157a1-config-volume\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:56.550817 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:56.550389    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 16:45:56.550817 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:56.550406    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557b8b5bcc-9tqwn: secret "image-registry-tls" not found
Jul 20 16:45:56.550817 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.550426    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4085a903-e1c6-4f41-9e07-66ea43060692-image-registry-private-configuration\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.550817 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:56.550465    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls podName:4085a903-e1c6-4f41-9e07-66ea43060692 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:57.050449376 +0000 UTC m=+36.598305419 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls") pod "image-registry-557b8b5bcc-9tqwn" (UID: "4085a903-e1c6-4f41-9e07-66ea43060692") : secret "image-registry-tls" not found
Jul 20 16:45:56.550817 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.550483    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4085a903-e1c6-4f41-9e07-66ea43060692-trusted-ca\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.550817 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.550519    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4085a903-e1c6-4f41-9e07-66ea43060692-registry-certificates\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.550817 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.550568    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/372d96f7-664c-4583-a4f2-5f4af55157a1-tmp-dir\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:56.550817 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.550633    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89b7q\" (UniqueName: \"kubernetes.io/projected/d5e37e15-6c9c-4b17-b026-5ae24cbe3e04-kube-api-access-89b7q\") pod \"ingress-canary-p2vpf\" (UID: \"d5e37e15-6c9c-4b17-b026-5ae24cbe3e04\") " pod="openshift-ingress-canary/ingress-canary-p2vpf"
Jul 20 16:45:56.550817 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.550657    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr2n2\" (UniqueName: \"kubernetes.io/projected/372d96f7-664c-4583-a4f2-5f4af55157a1-kube-api-access-pr2n2\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:56.550817 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.550680    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmr9m\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-kube-api-access-jmr9m\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.551218 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.551052    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4085a903-e1c6-4f41-9e07-66ea43060692-registry-certificates\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.551218 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.551085    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/372d96f7-664c-4583-a4f2-5f4af55157a1-tmp-dir\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:56.551218 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.551099    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5e37e15-6c9c-4b17-b026-5ae24cbe3e04-cert\") pod \"ingress-canary-p2vpf\" (UID: \"d5e37e15-6c9c-4b17-b026-5ae24cbe3e04\") " pod="openshift-ingress-canary/ingress-canary-p2vpf"
Jul 20 16:45:56.551218 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.551123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4085a903-e1c6-4f41-9e07-66ea43060692-ca-trust-extracted\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.551436 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.551415    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4085a903-e1c6-4f41-9e07-66ea43060692-ca-trust-extracted\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.551436 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.551428    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4085a903-e1c6-4f41-9e07-66ea43060692-trusted-ca\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.554574 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.554554    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4085a903-e1c6-4f41-9e07-66ea43060692-installation-pull-secrets\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.554666 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.554575    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5e37e15-6c9c-4b17-b026-5ae24cbe3e04-cert\") pod \"ingress-canary-p2vpf\" (UID: \"d5e37e15-6c9c-4b17-b026-5ae24cbe3e04\") " pod="openshift-ingress-canary/ingress-canary-p2vpf"
Jul 20 16:45:56.554666 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.554613    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4085a903-e1c6-4f41-9e07-66ea43060692-image-registry-private-configuration\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.559202 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.559178    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-bound-sa-token\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.559310 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.559280    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmr9m\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-kube-api-access-jmr9m\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:56.559658 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.559637    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr2n2\" (UniqueName: \"kubernetes.io/projected/372d96f7-664c-4583-a4f2-5f4af55157a1-kube-api-access-pr2n2\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:56.560372 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.560350    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89b7q\" (UniqueName: \"kubernetes.io/projected/d5e37e15-6c9c-4b17-b026-5ae24cbe3e04-kube-api-access-89b7q\") pod \"ingress-canary-p2vpf\" (UID: \"d5e37e15-6c9c-4b17-b026-5ae24cbe3e04\") " pod="openshift-ingress-canary/ingress-canary-p2vpf"
Jul 20 16:45:56.593491 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.593460    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:56.637556 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.637345    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p2vpf"
Jul 20 16:45:56.782469 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.782437    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p2vpf"]
Jul 20 16:45:56.893884 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.893862    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:45:56.896628 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.896593    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9vk4l\""
Jul 20 16:45:56.896628 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.896616    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 16:45:56.896824 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:56.896616    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 16:45:57.055690 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:57.055641    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:57.055690 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:57.055692    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:57.055927 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:57.055789    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 16:45:57.055927 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:57.055860    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls podName:372d96f7-664c-4583-a4f2-5f4af55157a1 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:58.055839537 +0000 UTC m=+37.603695593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls") pod "dns-default-m4g87" (UID: "372d96f7-664c-4583-a4f2-5f4af55157a1") : secret "dns-default-metrics-tls" not found
Jul 20 16:45:57.055927 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:57.055791    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 16:45:57.055927 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:57.055896    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557b8b5bcc-9tqwn: secret "image-registry-tls" not found
Jul 20 16:45:57.056137 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:57.055945    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls podName:4085a903-e1c6-4f41-9e07-66ea43060692 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:58.055932581 +0000 UTC m=+37.603788624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls") pod "image-registry-557b8b5bcc-9tqwn" (UID: "4085a903-e1c6-4f41-9e07-66ea43060692") : secret "image-registry-tls" not found
Jul 20 16:45:57.890865 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:57.890676    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:45:57.891027 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:57.890743    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:45:57.893706 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:57.893451    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 16:45:57.893706 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:57.893482    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 16:45:57.893706 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:57.893505    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2sqpl\""
Jul 20 16:45:58.014112 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:58.014079    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:58.064016 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:58.063984    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:45:58.064192 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:58.064028    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:45:58.064192 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:58.064131    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 16:45:58.064192 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:58.064141    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 16:45:58.064384 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:58.064215    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls podName:372d96f7-664c-4583-a4f2-5f4af55157a1 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:00.064195307 +0000 UTC m=+39.612051350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls") pod "dns-default-m4g87" (UID: "372d96f7-664c-4583-a4f2-5f4af55157a1") : secret "dns-default-metrics-tls" not found
Jul 20 16:45:58.064384 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:58.064144    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557b8b5bcc-9tqwn: secret "image-registry-tls" not found
Jul 20 16:45:58.064384 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:58.064330    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls podName:4085a903-e1c6-4f41-9e07-66ea43060692 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:00.064310774 +0000 UTC m=+39.612166819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls") pod "image-registry-557b8b5bcc-9tqwn" (UID: "4085a903-e1c6-4f41-9e07-66ea43060692") : secret "image-registry-tls" not found
Jul 20 16:45:58.116894 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:58.116860    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:58.208811 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:45:58.208718    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e37e15_6c9c_4b17_b026_5ae24cbe3e04.slice/crio-d4c47154b81a735f39f9c3e92cb20910f6f23fb814bf4920973db6db0524fd36 WatchSource:0}: Error finding container d4c47154b81a735f39f9c3e92cb20910f6f23fb814bf4920973db6db0524fd36: Status 404 returned error can't find the container with id d4c47154b81a735f39f9c3e92cb20910f6f23fb814bf4920973db6db0524fd36
Jul 20 16:45:58.884340 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:58.884311    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-d8b4g"]
Jul 20 16:45:58.887469 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:58.887443    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:58.890048 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:58.890027    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 20 16:45:58.890367 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:58.890345    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tcv9s\""
Jul 20 16:45:58.891487 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:58.891193    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 20 16:45:58.891487 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:58.891212    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 20 16:45:58.891487 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:58.891193    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 20 16:45:58.902682 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:58.902651    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d8b4g"]
Jul 20 16:45:58.938339 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:58.938307    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:59.054718 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.054683    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:59.070739 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.070707    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/30638c7b-0d95-4b98-947d-829c24bb32de-crio-socket\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.070858 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.070747    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/30638c7b-0d95-4b98-947d-829c24bb32de-data-volume\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.070858 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.070783    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs4mr\" (UniqueName: \"kubernetes.io/projected/30638c7b-0d95-4b98-947d-829c24bb32de-kube-api-access-zs4mr\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.070978 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.070891    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.070978 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.070926    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.147391 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.147309    2568 generic.go:356] "Generic (PLEG): container finished" podID="ff628bb3-d193-412f-bae5-c23349f488da" containerID="1fb57a5c6cb354279112033fe8f39133470d7862d8fc9c9bf0a1a709e017c2db" exitCode=0
Jul 20 16:45:59.147561 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.147388    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f844c" event={"ID":"ff628bb3-d193-412f-bae5-c23349f488da","Type":"ContainerDied","Data":"1fb57a5c6cb354279112033fe8f39133470d7862d8fc9c9bf0a1a709e017c2db"}
Jul 20 16:45:59.148616 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.148578    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p2vpf" event={"ID":"d5e37e15-6c9c-4b17-b026-5ae24cbe3e04","Type":"ContainerStarted","Data":"d4c47154b81a735f39f9c3e92cb20910f6f23fb814bf4920973db6db0524fd36"}
Jul 20 16:45:59.172211 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.171289    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.172211 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.171337    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.172211 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.171422    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/30638c7b-0d95-4b98-947d-829c24bb32de-crio-socket\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.172211 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.171449    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/30638c7b-0d95-4b98-947d-829c24bb32de-data-volume\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.172211 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.171481    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs4mr\" (UniqueName: \"kubernetes.io/projected/30638c7b-0d95-4b98-947d-829c24bb32de-kube-api-access-zs4mr\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.172211 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:59.171820    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 16:45:59.172211 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:59.171880    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm podName:30638c7b-0d95-4b98-947d-829c24bb32de nodeName:}" failed. No retries permitted until 2026-07-20 16:45:59.671861479 +0000 UTC m=+39.219717525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d8b4g" (UID: "30638c7b-0d95-4b98-947d-829c24bb32de") : configmap "kube-rbac-proxy" not found
Jul 20 16:45:59.172211 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.172093    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/30638c7b-0d95-4b98-947d-829c24bb32de-crio-socket\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.172211 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:59.172131    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 16:45:59.172211 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.172172    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/30638c7b-0d95-4b98-947d-829c24bb32de-data-volume\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.172211 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:59.172189    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls podName:30638c7b-0d95-4b98-947d-829c24bb32de nodeName:}" failed. No retries permitted until 2026-07-20 16:45:59.672171237 +0000 UTC m=+39.220027300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d8b4g" (UID: "30638c7b-0d95-4b98-947d-829c24bb32de") : secret "insights-runtime-extractor-tls" not found
Jul 20 16:45:59.174388 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.174366    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:59.182931 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.182895    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs4mr\" (UniqueName: \"kubernetes.io/projected/30638c7b-0d95-4b98-947d-829c24bb32de-kube-api-access-zs4mr\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.249143 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.249119    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:59.372873 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.372686    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:59.465524 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.465441    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:59.598847 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.598805    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:59.619658 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.619629    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tclrk"]
Jul 20 16:45:59.622877 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.622855    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tclrk"
Jul 20 16:45:59.625517 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.625494    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-wz9fv\""
Jul 20 16:45:59.625631 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.625536    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 20 16:45:59.625631 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.625494    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 20 16:45:59.631197 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.631175    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tclrk"]
Jul 20 16:45:59.674813 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.674775    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.674971 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.674819    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:45:59.674971 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:59.674897    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 16:45:59.674971 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:59.674960    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm podName:30638c7b-0d95-4b98-947d-829c24bb32de nodeName:}" failed. No retries permitted until 2026-07-20 16:46:00.674942481 +0000 UTC m=+40.222798541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d8b4g" (UID: "30638c7b-0d95-4b98-947d-829c24bb32de") : configmap "kube-rbac-proxy" not found
Jul 20 16:45:59.675142 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:59.674982    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 16:45:59.675142 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:45:59.675035    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls podName:30638c7b-0d95-4b98-947d-829c24bb32de nodeName:}" failed. No retries permitted until 2026-07-20 16:46:00.67502423 +0000 UTC m=+40.222880287 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d8b4g" (UID: "30638c7b-0d95-4b98-947d-829c24bb32de") : secret "insights-runtime-extractor-tls" not found
Jul 20 16:45:59.692089 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.692055    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:59.745233 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.745163    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:59.775516 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.775490    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p8t8\" (UniqueName: \"kubernetes.io/projected/2106a93a-fea7-4adb-9933-46cb91da712d-kube-api-access-6p8t8\") pod \"migrator-7cc9c7ffd7-tclrk\" (UID: \"2106a93a-fea7-4adb-9933-46cb91da712d\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tclrk"
Jul 20 16:45:59.876185 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.876159    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p8t8\" (UniqueName: \"kubernetes.io/projected/2106a93a-fea7-4adb-9933-46cb91da712d-kube-api-access-6p8t8\") pod \"migrator-7cc9c7ffd7-tclrk\" (UID: \"2106a93a-fea7-4adb-9933-46cb91da712d\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tclrk"
Jul 20 16:45:59.884876 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.884849    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p8t8\" (UniqueName: \"kubernetes.io/projected/2106a93a-fea7-4adb-9933-46cb91da712d-kube-api-access-6p8t8\") pod \"migrator-7cc9c7ffd7-tclrk\" (UID: \"2106a93a-fea7-4adb-9933-46cb91da712d\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tclrk"
Jul 20 16:45:59.932359 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:45:59.932328    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tclrk"
Jul 20 16:46:00.078401 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:00.078324    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:46:00.078401 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:00.078360    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:46:00.078874 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:00.078482    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 16:46:00.078874 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:00.078494    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557b8b5bcc-9tqwn: secret "image-registry-tls" not found
Jul 20 16:46:00.078874 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:00.078494    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 16:46:00.078874 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:00.078550    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls podName:4085a903-e1c6-4f41-9e07-66ea43060692 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:04.078531342 +0000 UTC m=+43.626387409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls") pod "image-registry-557b8b5bcc-9tqwn" (UID: "4085a903-e1c6-4f41-9e07-66ea43060692") : secret "image-registry-tls" not found
Jul 20 16:46:00.078874 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:00.078564    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls podName:372d96f7-664c-4583-a4f2-5f4af55157a1 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:04.078556729 +0000 UTC m=+43.626412773 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls") pod "dns-default-m4g87" (UID: "372d96f7-664c-4583-a4f2-5f4af55157a1") : secret "dns-default-metrics-tls" not found
Jul 20 16:46:00.155078 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:00.155043    2568 generic.go:356] "Generic (PLEG): container finished" podID="ff628bb3-d193-412f-bae5-c23349f488da" containerID="ad35ef896376ff7e8fe5b30b6bca267964c659a135669d0c3587662db7ad05a1" exitCode=0
Jul 20 16:46:00.155259 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:00.155111    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f844c" event={"ID":"ff628bb3-d193-412f-bae5-c23349f488da","Type":"ContainerDied","Data":"ad35ef896376ff7e8fe5b30b6bca267964c659a135669d0c3587662db7ad05a1"}
Jul 20 16:46:00.263485 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:00.263450    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6g5w2_798679f7-4546-4546-bc3d-c12693574272/dns-node-resolver/0.log"
Jul 20 16:46:00.684098 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:00.684067    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:46:00.684272 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:00.684114    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:46:00.684272 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:00.684199    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 16:46:00.684389 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:00.684280    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 16:46:00.684389 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:00.684291    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm podName:30638c7b-0d95-4b98-947d-829c24bb32de nodeName:}" failed. No retries permitted until 2026-07-20 16:46:02.684269137 +0000 UTC m=+42.232125201 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d8b4g" (UID: "30638c7b-0d95-4b98-947d-829c24bb32de") : configmap "kube-rbac-proxy" not found
Jul 20 16:46:00.684389 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:00.684339    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls podName:30638c7b-0d95-4b98-947d-829c24bb32de nodeName:}" failed. No retries permitted until 2026-07-20 16:46:02.684319832 +0000 UTC m=+42.232175878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d8b4g" (UID: "30638c7b-0d95-4b98-947d-829c24bb32de") : secret "insights-runtime-extractor-tls" not found
Jul 20 16:46:00.749056 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:00.749029    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:00.757913 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:00.757893    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tclrk"]
Jul 20 16:46:00.760742 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:00.760720    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2106a93a_fea7_4adb_9933_46cb91da712d.slice/crio-0407aa7f76eadd2a94b39052cc7152f747c6b92614e8fc2c90bd08ce0e3ee854 WatchSource:0}: Error finding container 0407aa7f76eadd2a94b39052cc7152f747c6b92614e8fc2c90bd08ce0e3ee854: Status 404 returned error can't find the container with id 0407aa7f76eadd2a94b39052cc7152f747c6b92614e8fc2c90bd08ce0e3ee854
Jul 20 16:46:00.849278 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:00.849223    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:01.010333 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.010308    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:01.159013 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.158983    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tclrk" event={"ID":"2106a93a-fea7-4adb-9933-46cb91da712d","Type":"ContainerStarted","Data":"0407aa7f76eadd2a94b39052cc7152f747c6b92614e8fc2c90bd08ce0e3ee854"}
Jul 20 16:46:01.160656 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.160621    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p2vpf" event={"ID":"d5e37e15-6c9c-4b17-b026-5ae24cbe3e04","Type":"ContainerStarted","Data":"5d50dde3a0408c8f384f73b13411c9af3715a34a3a149ed3cabe2117d6faceee"}
Jul 20 16:46:01.164673 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.164651    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f844c" event={"ID":"ff628bb3-d193-412f-bae5-c23349f488da","Type":"ContainerStarted","Data":"bbcdfdae26a8fe0626c9f13bbc9243e26165b69c9909b7d8ae7d46242a8aa29b"}
Jul 20 16:46:01.178321 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.178273    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p2vpf" podStartSLOduration=2.383178878 podStartE2EDuration="5.178259903s" totalImagesPullingTime="2.795081025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:01.176647067 +0000 UTC m=+40.724503137" watchObservedRunningTime="2026-07-20 16:46:01.178259903 +0000 UTC m=+40.726115961"
Jul 20 16:46:01.201670 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.201620    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-f844c" podStartSLOduration=8.818392953 podStartE2EDuration="41.201599594s" totalImagesPullingTime="32.383206641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:01.199293923 +0000 UTC m=+40.747149988" watchObservedRunningTime="2026-07-20 16:46:01.201599594 +0000 UTC m=+40.749455661"
Jul 20 16:46:01.322955 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.322844    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:01.422211 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.422174    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:01.653255 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.653197    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-wzjvb"]
Jul 20 16:46:01.666103 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.666077    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sfzwq_46c16140-8350-4ab4-9e33-b92a7fb16f1e/node-ca/0.log"
Jul 20 16:46:01.686609 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.686556    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:01.687805 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.687783    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-wzjvb"]
Jul 20 16:46:01.687894 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.687886    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-wzjvb"
Jul 20 16:46:01.690555 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.690364    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 20 16:46:01.690555 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.690427    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 20 16:46:01.690555 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.690433    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 20 16:46:01.690555 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.690457    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-jzj65\""
Jul 20 16:46:01.690778 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.690567    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 20 16:46:01.742962 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.742935    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:01.792182 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.792149    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxvgq\" (UniqueName: \"kubernetes.io/projected/cefc6381-c29e-4453-989d-e168e76bc16d-kube-api-access-bxvgq\") pod \"service-ca-58598695f7-wzjvb\" (UID: \"cefc6381-c29e-4453-989d-e168e76bc16d\") " pod="openshift-service-ca/service-ca-58598695f7-wzjvb"
Jul 20 16:46:01.792330 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.792311    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cefc6381-c29e-4453-989d-e168e76bc16d-signing-key\") pod \"service-ca-58598695f7-wzjvb\" (UID: \"cefc6381-c29e-4453-989d-e168e76bc16d\") " pod="openshift-service-ca/service-ca-58598695f7-wzjvb"
Jul 20 16:46:01.792367 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.792348    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cefc6381-c29e-4453-989d-e168e76bc16d-signing-cabundle\") pod \"service-ca-58598695f7-wzjvb\" (UID: \"cefc6381-c29e-4453-989d-e168e76bc16d\") " pod="openshift-service-ca/service-ca-58598695f7-wzjvb"
Jul 20 16:46:01.893803 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.893772    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cefc6381-c29e-4453-989d-e168e76bc16d-signing-key\") pod \"service-ca-58598695f7-wzjvb\" (UID: \"cefc6381-c29e-4453-989d-e168e76bc16d\") " pod="openshift-service-ca/service-ca-58598695f7-wzjvb"
Jul 20 16:46:01.893803 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.893807    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cefc6381-c29e-4453-989d-e168e76bc16d-signing-cabundle\") pod \"service-ca-58598695f7-wzjvb\" (UID: \"cefc6381-c29e-4453-989d-e168e76bc16d\") " pod="openshift-service-ca/service-ca-58598695f7-wzjvb"
Jul 20 16:46:01.894043 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.893862    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxvgq\" (UniqueName: \"kubernetes.io/projected/cefc6381-c29e-4453-989d-e168e76bc16d-kube-api-access-bxvgq\") pod \"service-ca-58598695f7-wzjvb\" (UID: \"cefc6381-c29e-4453-989d-e168e76bc16d\") " pod="openshift-service-ca/service-ca-58598695f7-wzjvb"
Jul 20 16:46:01.894732 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.894693    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cefc6381-c29e-4453-989d-e168e76bc16d-signing-cabundle\") pod \"service-ca-58598695f7-wzjvb\" (UID: \"cefc6381-c29e-4453-989d-e168e76bc16d\") " pod="openshift-service-ca/service-ca-58598695f7-wzjvb"
Jul 20 16:46:01.896618 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.896597    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cefc6381-c29e-4453-989d-e168e76bc16d-signing-key\") pod \"service-ca-58598695f7-wzjvb\" (UID: \"cefc6381-c29e-4453-989d-e168e76bc16d\") " pod="openshift-service-ca/service-ca-58598695f7-wzjvb"
Jul 20 16:46:01.903153 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.903128    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxvgq\" (UniqueName: \"kubernetes.io/projected/cefc6381-c29e-4453-989d-e168e76bc16d-kube-api-access-bxvgq\") pod \"service-ca-58598695f7-wzjvb\" (UID: \"cefc6381-c29e-4453-989d-e168e76bc16d\") " pod="openshift-service-ca/service-ca-58598695f7-wzjvb"
Jul 20 16:46:01.998313 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:01.998217    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-wzjvb"
Jul 20 16:46:02.205501 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:02.205472    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-wzjvb"]
Jul 20 16:46:02.208953 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:02.208922    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcefc6381_c29e_4453_989d_e168e76bc16d.slice/crio-b7dabdc5d6ee3726c237490bc01d6e0681611f003db7b838640b53363d039764 WatchSource:0}: Error finding container b7dabdc5d6ee3726c237490bc01d6e0681611f003db7b838640b53363d039764: Status 404 returned error can't find the container with id b7dabdc5d6ee3726c237490bc01d6e0681611f003db7b838640b53363d039764
Jul 20 16:46:02.269644 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:02.269578    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p2vpf_d5e37e15-6c9c-4b17-b026-5ae24cbe3e04/serve-healthcheck-canary/0.log"
Jul 20 16:46:02.710676 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:02.710645    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:46:02.710676 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:02.710677    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:46:02.710870 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:02.710775    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 16:46:02.710870 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:02.710794    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 16:46:02.710870 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:02.710839    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls podName:30638c7b-0d95-4b98-947d-829c24bb32de nodeName:}" failed. No retries permitted until 2026-07-20 16:46:06.710825901 +0000 UTC m=+46.258681944 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d8b4g" (UID: "30638c7b-0d95-4b98-947d-829c24bb32de") : secret "insights-runtime-extractor-tls" not found
Jul 20 16:46:02.710870 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:02.710850    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm podName:30638c7b-0d95-4b98-947d-829c24bb32de nodeName:}" failed. No retries permitted until 2026-07-20 16:46:06.710844464 +0000 UTC m=+46.258700508 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d8b4g" (UID: "30638c7b-0d95-4b98-947d-829c24bb32de") : configmap "kube-rbac-proxy" not found
Jul 20 16:46:02.761919 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:02.761863    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:03.173157 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:03.172708    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tclrk" event={"ID":"2106a93a-fea7-4adb-9933-46cb91da712d","Type":"ContainerStarted","Data":"2c977f8ed70b1339296406b75a35104bbcf476814d09e8af0618884ec62755b5"}
Jul 20 16:46:03.173157 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:03.172773    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tclrk" event={"ID":"2106a93a-fea7-4adb-9933-46cb91da712d","Type":"ContainerStarted","Data":"1c964d0b737f3c5e2dd523d27911c5129d1312aafc65fe80ea8f230abaeac445"}
Jul 20 16:46:03.173781 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:03.173757    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-wzjvb" event={"ID":"cefc6381-c29e-4453-989d-e168e76bc16d","Type":"ContainerStarted","Data":"b7dabdc5d6ee3726c237490bc01d6e0681611f003db7b838640b53363d039764"}
Jul 20 16:46:03.191187 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:03.191110    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:03.191817 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:03.191781    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-tclrk" podStartSLOduration=2.515106623 podStartE2EDuration="4.191767482s" totalImagesPullingTime="1.676660859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:03.190736961 +0000 UTC m=+42.738593027" watchObservedRunningTime="2026-07-20 16:46:03.191767482 +0000 UTC m=+42.739623548"
Jul 20 16:46:04.123257 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:04.123207    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:46:04.123257 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:04.123261    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:46:04.123756 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:04.123360    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 16:46:04.123756 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:04.123430    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 16:46:04.123756 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:04.123449    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557b8b5bcc-9tqwn: secret "image-registry-tls" not found
Jul 20 16:46:04.123756 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:04.123432    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls podName:372d96f7-664c-4583-a4f2-5f4af55157a1 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:12.12341076 +0000 UTC m=+51.671266806 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls") pod "dns-default-m4g87" (UID: "372d96f7-664c-4583-a4f2-5f4af55157a1") : secret "dns-default-metrics-tls" not found
Jul 20 16:46:04.123756 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:04.123517    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls podName:4085a903-e1c6-4f41-9e07-66ea43060692 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:12.123496896 +0000 UTC m=+51.671352945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls") pod "image-registry-557b8b5bcc-9tqwn" (UID: "4085a903-e1c6-4f41-9e07-66ea43060692") : secret "image-registry-tls" not found
Jul 20 16:46:05.180312 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:05.180278    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-wzjvb" event={"ID":"cefc6381-c29e-4453-989d-e168e76bc16d","Type":"ContainerStarted","Data":"7172f99d12662e91bc2116738b2146fd8d3e22d2487b91978ad2de05e2b8555b"}
Jul 20 16:46:05.196258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:05.196188    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-wzjvb" podStartSLOduration=1.708744241 podStartE2EDuration="4.19617345s" totalImagesPullingTime="2.487429209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:05.194960247 +0000 UTC m=+44.742816313" watchObservedRunningTime="2026-07-20 16:46:05.19617345 +0000 UTC m=+44.744029539"
Jul 20 16:46:05.571854 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:05.571756    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:05.902300 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:05.902268    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:06.100048 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:06.100009    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:06.200388 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:06.200298    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:06.302168 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:06.302130    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:06.402016 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:06.401977    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:06.743161 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:06.743126    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:46:06.743161 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:06.743160    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:46:06.743388 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:06.743280    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 16:46:06.743388 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:06.743332    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls podName:30638c7b-0d95-4b98-947d-829c24bb32de nodeName:}" failed. No retries permitted until 2026-07-20 16:46:14.743319906 +0000 UTC m=+54.291175950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d8b4g" (UID: "30638c7b-0d95-4b98-947d-829c24bb32de") : secret "insights-runtime-extractor-tls" not found
Jul 20 16:46:06.743643 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:06.743626    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30638c7b-0d95-4b98-947d-829c24bb32de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:46:06.800750 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:06.800714    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:06.900800 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:06.900760    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:06.960304 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:06.960265    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:07.396838 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:07.396803    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:07.795868 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:07.795780    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:08.210500 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:08.210462    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:08.399981 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:08.399940    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:08.798654 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:08.798616    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:09.036515 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:09.036479    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:09.399913 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:09.399855    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:09.600702 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:09.600669    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:09.726064 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:09.725985    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:09.995725 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:09.995642    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:11.162604 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:11.162571    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:11.263518 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:11.263483    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:11.602357 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:11.602272    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:11.992684 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:11.992641    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:12.185773 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:12.185746    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:46:12.186233 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:12.185780    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:46:12.188209 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:12.188176    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/372d96f7-664c-4583-a4f2-5f4af55157a1-metrics-tls\") pod \"dns-default-m4g87\" (UID: \"372d96f7-664c-4583-a4f2-5f4af55157a1\") " pod="openshift-dns/dns-default-m4g87"
Jul 20 16:46:12.188351 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:12.188308    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls\") pod \"image-registry-557b8b5bcc-9tqwn\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") " pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:46:12.190890 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:12.190865    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:12.220988 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:12.220963    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:46:12.243144 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:12.243066    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m4g87"
Jul 20 16:46:12.289623 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:12.289592    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:12.382929 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:12.382901    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-557b8b5bcc-9tqwn"]
Jul 20 16:46:12.386029 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:12.386005    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4085a903_e1c6_4f41_9e07_66ea43060692.slice/crio-eb50dadc25c20c9c9642d78a0dd7831ac5dee5912f297614136a883e9fb8d0fd WatchSource:0}: Error finding container eb50dadc25c20c9c9642d78a0dd7831ac5dee5912f297614136a883e9fb8d0fd: Status 404 returned error can't find the container with id eb50dadc25c20c9c9642d78a0dd7831ac5dee5912f297614136a883e9fb8d0fd
Jul 20 16:46:12.395081 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:12.395059    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m4g87"]
Jul 20 16:46:12.397883 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:12.397854    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372d96f7_664c_4583_a4f2_5f4af55157a1.slice/crio-58edc627b034c87b692790aa78297aa0de9bf07cb2e0accd535538b2b3a3bbd0 WatchSource:0}: Error finding container 58edc627b034c87b692790aa78297aa0de9bf07cb2e0accd535538b2b3a3bbd0: Status 404 returned error can't find the container with id 58edc627b034c87b692790aa78297aa0de9bf07cb2e0accd535538b2b3a3bbd0
Jul 20 16:46:12.398669 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:12.398644    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:12.603449 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:12.603399    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:13.202724 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:13.202684    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4g87" event={"ID":"372d96f7-664c-4583-a4f2-5f4af55157a1","Type":"ContainerStarted","Data":"58edc627b034c87b692790aa78297aa0de9bf07cb2e0accd535538b2b3a3bbd0"}
Jul 20 16:46:13.204340 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:13.204314    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn" event={"ID":"4085a903-e1c6-4f41-9e07-66ea43060692","Type":"ContainerStarted","Data":"60f3bc00a47a7b4b3cfc1fb43e300a4d949753f25ab9dc6bec37c9f4acd5b1e9"}
Jul 20 16:46:13.204635 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:13.204354    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn" event={"ID":"4085a903-e1c6-4f41-9e07-66ea43060692","Type":"ContainerStarted","Data":"eb50dadc25c20c9c9642d78a0dd7831ac5dee5912f297614136a883e9fb8d0fd"}
Jul 20 16:46:13.204635 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:13.204442    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:46:13.224073 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:13.224024    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn" podStartSLOduration=24.224010926 podStartE2EDuration="24.224010926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:13.222768983 +0000 UTC m=+52.770625049" watchObservedRunningTime="2026-07-20 16:46:13.224010926 +0000 UTC m=+52.771866992"
Jul 20 16:46:13.608302 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:13.608195    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:13.797433 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:13.797402    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:14.209861 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:14.209819    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:14.393332 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:14.393297    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:14.807083 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:14.807048    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:46:14.809920 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:14.809888    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30638c7b-0d95-4b98-947d-829c24bb32de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d8b4g\" (UID: \"30638c7b-0d95-4b98-947d-829c24bb32de\") " pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:46:14.993673 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:14.993587    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:15.098623 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:15.098543    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d8b4g"
Jul 20 16:46:15.233003 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:15.232957    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d8b4g"]
Jul 20 16:46:15.235458 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:15.235434    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30638c7b_0d95_4b98_947d_829c24bb32de.slice/crio-605734381bf549fdbdf058b9cb5fd025415f1caf2ca02c4b41d721331dc9b2e0 WatchSource:0}: Error finding container 605734381bf549fdbdf058b9cb5fd025415f1caf2ca02c4b41d721331dc9b2e0: Status 404 returned error can't find the container with id 605734381bf549fdbdf058b9cb5fd025415f1caf2ca02c4b41d721331dc9b2e0
Jul 20 16:46:15.396760 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:15.396728    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:15.794258 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:15.794176    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:15.999683 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:15.999642    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:16.218445 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:16.218408    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4g87" event={"ID":"372d96f7-664c-4583-a4f2-5f4af55157a1","Type":"ContainerStarted","Data":"e97264834476f487252619bec9e6168312625a3cde217517482207775531b527"}
Jul 20 16:46:16.218620 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:16.218507    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4g87" event={"ID":"372d96f7-664c-4583-a4f2-5f4af55157a1","Type":"ContainerStarted","Data":"eb674652201c2a4746125ee468294645e0695c1dadc4320d1aa022ab1e03e420"}
Jul 20 16:46:16.218620 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:16.218532    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-m4g87"
Jul 20 16:46:16.220098 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:16.220072    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d8b4g" event={"ID":"30638c7b-0d95-4b98-947d-829c24bb32de","Type":"ContainerStarted","Data":"605734381bf549fdbdf058b9cb5fd025415f1caf2ca02c4b41d721331dc9b2e0"}
Jul 20 16:46:16.236083 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:16.236039    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-m4g87" podStartSLOduration=16.795849819 podStartE2EDuration="20.236026166s" totalImagesPullingTime="3.440176347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:16.235505799 +0000 UTC m=+55.783361866" watchObservedRunningTime="2026-07-20 16:46:16.236026166 +0000 UTC m=+55.783882232"
Jul 20 16:46:16.393110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:16.393061    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:16.594597 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:16.594562    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:21.239060 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.239022    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d8b4g" event={"ID":"30638c7b-0d95-4b98-947d-829c24bb32de","Type":"ContainerStarted","Data":"a746365956aea723022cb680df1e8333cbd8f4b61583e4a77d43fc8398bc7b95"}
Jul 20 16:46:21.595756 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.595666    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh"]
Jul 20 16:46:21.613968 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.613932    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"]
Jul 20 16:46:21.614150 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.614109    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh"
Jul 20 16:46:21.617362 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.617314    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 20 16:46:21.617595 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.617581    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 20 16:46:21.617976 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.617933    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-txssd\""
Jul 20 16:46:21.628111 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.628089    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl"]
Jul 20 16:46:21.628226 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.628205    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"
Jul 20 16:46:21.630922 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.630903    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 20 16:46:21.632160 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.631801    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 20 16:46:21.632160 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.632020    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 20 16:46:21.634706 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.634688    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 20 16:46:21.644861 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.644842    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"]
Jul 20 16:46:21.644999 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.644982    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl"
Jul 20 16:46:21.647768 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.647746    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 20 16:46:21.648064 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.648047    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-lq6hx\""
Jul 20 16:46:21.664121 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.664006    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/148b5994-e9e4-40d5-8192-2382f30f57e0-klusterlet-config\") pod \"klusterlet-addon-workmgr-f59df4798-86tmv\" (UID: \"148b5994-e9e4-40d5-8192-2382f30f57e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"
Jul 20 16:46:21.664121 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.664042    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c63ca7b6-b603-474b-bc8a-66d0d0fba0f9-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-f4bbh\" (UID: \"c63ca7b6-b603-474b-bc8a-66d0d0fba0f9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh"
Jul 20 16:46:21.664121 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.664117    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/148b5994-e9e4-40d5-8192-2382f30f57e0-tmp\") pod \"klusterlet-addon-workmgr-f59df4798-86tmv\" (UID: \"148b5994-e9e4-40d5-8192-2382f30f57e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"
Jul 20 16:46:21.664395 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.664205    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrgsj\" (UniqueName: \"kubernetes.io/projected/148b5994-e9e4-40d5-8192-2382f30f57e0-kube-api-access-hrgsj\") pod \"klusterlet-addon-workmgr-f59df4798-86tmv\" (UID: \"148b5994-e9e4-40d5-8192-2382f30f57e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"
Jul 20 16:46:21.664395 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.664234    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c63ca7b6-b603-474b-bc8a-66d0d0fba0f9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-f4bbh\" (UID: \"c63ca7b6-b603-474b-bc8a-66d0d0fba0f9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh"
Jul 20 16:46:21.664806 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.664788    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh"]
Jul 20 16:46:21.664887 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.664820    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl"]
Jul 20 16:46:21.664887 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.664831    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"]
Jul 20 16:46:21.664887 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.664850    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"]
Jul 20 16:46:21.664887 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.664876    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-557b8b5bcc-9tqwn"]
Jul 20 16:46:21.665070 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.664928    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.667766 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.667739    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 20 16:46:21.667997 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.667969    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 20 16:46:21.668927 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.668906    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 20 16:46:21.669062 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.669039    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 20 16:46:21.713578 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.713549    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f6f846bcf-jgw5v"]
Jul 20 16:46:21.724047 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.724021    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.737550 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.736958    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f6f846bcf-jgw5v"]
Jul 20 16:46:21.754922 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.754882    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:21.764705 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.764678    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8edb7145-cd73-4a73-9383-5d229747f573-ca\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.764854 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.764729    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-image-registry-private-configuration\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.764854 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.764807    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/84ffed29-a741-4bad-a0c1-16e8f4596acc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-dc95df6b-mdmwl\" (UID: \"84ffed29-a741-4bad-a0c1-16e8f4596acc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl"
Jul 20 16:46:21.764854 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.764832    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57tt4\" (UniqueName: \"kubernetes.io/projected/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-kube-api-access-57tt4\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.764988 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.764857    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87vgh\" (UniqueName: \"kubernetes.io/projected/8edb7145-cd73-4a73-9383-5d229747f573-kube-api-access-87vgh\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.764988 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.764886    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-registry-certificates\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.764988 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.764916    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8edb7145-cd73-4a73-9383-5d229747f573-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.764988 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.764947    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/148b5994-e9e4-40d5-8192-2382f30f57e0-klusterlet-config\") pod \"klusterlet-addon-workmgr-f59df4798-86tmv\" (UID: \"148b5994-e9e4-40d5-8192-2382f30f57e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"
Jul 20 16:46:21.764988 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.764963    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c63ca7b6-b603-474b-bc8a-66d0d0fba0f9-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-f4bbh\" (UID: \"c63ca7b6-b603-474b-bc8a-66d0d0fba0f9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh"
Jul 20 16:46:21.764988 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.764986    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-trusted-ca\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.765360 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.765011    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-bound-sa-token\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.765360 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.765042    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/148b5994-e9e4-40d5-8192-2382f30f57e0-tmp\") pod \"klusterlet-addon-workmgr-f59df4798-86tmv\" (UID: \"148b5994-e9e4-40d5-8192-2382f30f57e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"
Jul 20 16:46:21.765360 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.765089    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8edb7145-cd73-4a73-9383-5d229747f573-additional-service-ca\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.765360 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.765168    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8edb7145-cd73-4a73-9383-5d229747f573-hub\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.765360 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.765204    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-registry-tls\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.765360 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.765284    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrgsj\" (UniqueName: \"kubernetes.io/projected/148b5994-e9e4-40d5-8192-2382f30f57e0-kube-api-access-hrgsj\") pod \"klusterlet-addon-workmgr-f59df4798-86tmv\" (UID: \"148b5994-e9e4-40d5-8192-2382f30f57e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"
Jul 20 16:46:21.765626 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.765397    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/148b5994-e9e4-40d5-8192-2382f30f57e0-tmp\") pod \"klusterlet-addon-workmgr-f59df4798-86tmv\" (UID: \"148b5994-e9e4-40d5-8192-2382f30f57e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"
Jul 20 16:46:21.765626 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.765407    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c63ca7b6-b603-474b-bc8a-66d0d0fba0f9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-f4bbh\" (UID: \"c63ca7b6-b603-474b-bc8a-66d0d0fba0f9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh"
Jul 20 16:46:21.765626 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.765453    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8edb7145-cd73-4a73-9383-5d229747f573-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.765626 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.765481    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-ca-trust-extracted\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.765626 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.765614    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-275b8\" (UniqueName: \"kubernetes.io/projected/84ffed29-a741-4bad-a0c1-16e8f4596acc-kube-api-access-275b8\") pod \"managed-serviceaccount-addon-agent-dc95df6b-mdmwl\" (UID: \"84ffed29-a741-4bad-a0c1-16e8f4596acc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl"
Jul 20 16:46:21.765801 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.765636    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-installation-pull-secrets\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.765801 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.765712    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c63ca7b6-b603-474b-bc8a-66d0d0fba0f9-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-f4bbh\" (UID: \"c63ca7b6-b603-474b-bc8a-66d0d0fba0f9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh"
Jul 20 16:46:21.767938 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.767911    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/148b5994-e9e4-40d5-8192-2382f30f57e0-klusterlet-config\") pod \"klusterlet-addon-workmgr-f59df4798-86tmv\" (UID: \"148b5994-e9e4-40d5-8192-2382f30f57e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"
Jul 20 16:46:21.768070 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.768052    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c63ca7b6-b603-474b-bc8a-66d0d0fba0f9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-f4bbh\" (UID: \"c63ca7b6-b603-474b-bc8a-66d0d0fba0f9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh"
Jul 20 16:46:21.774177 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.774153    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrgsj\" (UniqueName: \"kubernetes.io/projected/148b5994-e9e4-40d5-8192-2382f30f57e0-kube-api-access-hrgsj\") pod \"klusterlet-addon-workmgr-f59df4798-86tmv\" (UID: \"148b5994-e9e4-40d5-8192-2382f30f57e0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"
Jul 20 16:46:21.845861 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.845782    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:21.866362 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866329    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8edb7145-cd73-4a73-9383-5d229747f573-hub\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.866362 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866364    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-registry-tls\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.866599 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866391    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8edb7145-cd73-4a73-9383-5d229747f573-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.866599 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866407    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-ca-trust-extracted\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.866599 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866425    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-275b8\" (UniqueName: \"kubernetes.io/projected/84ffed29-a741-4bad-a0c1-16e8f4596acc-kube-api-access-275b8\") pod \"managed-serviceaccount-addon-agent-dc95df6b-mdmwl\" (UID: \"84ffed29-a741-4bad-a0c1-16e8f4596acc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl"
Jul 20 16:46:21.866599 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866447    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-installation-pull-secrets\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.866599 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866560    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8edb7145-cd73-4a73-9383-5d229747f573-ca\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.866841 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866623    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-image-registry-private-configuration\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.866841 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866678    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/84ffed29-a741-4bad-a0c1-16e8f4596acc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-dc95df6b-mdmwl\" (UID: \"84ffed29-a741-4bad-a0c1-16e8f4596acc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl"
Jul 20 16:46:21.866841 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866707    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57tt4\" (UniqueName: \"kubernetes.io/projected/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-kube-api-access-57tt4\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.866841 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866739    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87vgh\" (UniqueName: \"kubernetes.io/projected/8edb7145-cd73-4a73-9383-5d229747f573-kube-api-access-87vgh\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.866841 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866767    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-registry-certificates\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.866841 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866795    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8edb7145-cd73-4a73-9383-5d229747f573-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.866841 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866810    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-ca-trust-extracted\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.866841 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866820    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-trusted-ca\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.867262 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866847    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-bound-sa-token\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.867262 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.866889    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8edb7145-cd73-4a73-9383-5d229747f573-additional-service-ca\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.867676 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.867651    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8edb7145-cd73-4a73-9383-5d229747f573-additional-service-ca\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.869063 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.868993    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-trusted-ca\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.869376 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.869353    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-image-registry-private-configuration\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.870218 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.869648    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-installation-pull-secrets\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.870218 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.870132    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8edb7145-cd73-4a73-9383-5d229747f573-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.870218 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.870162    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-registry-certificates\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.870447 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.870317    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/84ffed29-a741-4bad-a0c1-16e8f4596acc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-dc95df6b-mdmwl\" (UID: \"84ffed29-a741-4bad-a0c1-16e8f4596acc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl"
Jul 20 16:46:21.870509 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.870484    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8edb7145-cd73-4a73-9383-5d229747f573-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.870595 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.870574    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8edb7145-cd73-4a73-9383-5d229747f573-hub\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.870936 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.870897    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-registry-tls\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.871698 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.871678    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8edb7145-cd73-4a73-9383-5d229747f573-ca\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.887554 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.887522    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-275b8\" (UniqueName: \"kubernetes.io/projected/84ffed29-a741-4bad-a0c1-16e8f4596acc-kube-api-access-275b8\") pod \"managed-serviceaccount-addon-agent-dc95df6b-mdmwl\" (UID: \"84ffed29-a741-4bad-a0c1-16e8f4596acc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl"
Jul 20 16:46:21.887968 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.887947    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87vgh\" (UniqueName: \"kubernetes.io/projected/8edb7145-cd73-4a73-9383-5d229747f573-kube-api-access-87vgh\") pod \"cluster-proxy-proxy-agent-564cc95d8d-g28dn\" (UID: \"8edb7145-cd73-4a73-9383-5d229747f573\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:21.888071 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.888031    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57tt4\" (UniqueName: \"kubernetes.io/projected/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-kube-api-access-57tt4\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.888124 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.888074    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c9724d1-ce8b-46de-b7fb-851f8aba5b98-bound-sa-token\") pod \"image-registry-6f6f846bcf-jgw5v\" (UID: \"7c9724d1-ce8b-46de-b7fb-851f8aba5b98\") " pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:21.927051 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.927028    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh"
Jul 20 16:46:21.940022 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.939977    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:21.941711 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.941693    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"
Jul 20 16:46:21.963146 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.963128    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl"
Jul 20 16:46:21.975546 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:21.975527    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"
Jul 20 16:46:22.048909 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.048734    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:22.058542 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.058230    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:22.131574 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.127547    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh"]
Jul 20 16:46:22.133576 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:22.133544    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63ca7b6_b603_474b_bc8a_66d0d0fba0f9.slice/crio-e5838310afc37607ac561fdee51742f7072b28c7313fef3fd33178c6ec56b9c8 WatchSource:0}: Error finding container e5838310afc37607ac561fdee51742f7072b28c7313fef3fd33178c6ec56b9c8: Status 404 returned error can't find the container with id e5838310afc37607ac561fdee51742f7072b28c7313fef3fd33178c6ec56b9c8
Jul 20 16:46:22.152567 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.152491    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:22.155534 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.155441    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-44gfk"
Jul 20 16:46:22.172678 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.172233    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"]
Jul 20 16:46:22.173653 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:22.173574    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod148b5994_e9e4_40d5_8192_2382f30f57e0.slice/crio-bf7dc0eca6a557a8ed58bec3e787e5882effaa3b4257810756da6e3ef65a2ec7 WatchSource:0}: Error finding container bf7dc0eca6a557a8ed58bec3e787e5882effaa3b4257810756da6e3ef65a2ec7: Status 404 returned error can't find the container with id bf7dc0eca6a557a8ed58bec3e787e5882effaa3b4257810756da6e3ef65a2ec7
Jul 20 16:46:22.187272 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.187190    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl"]
Jul 20 16:46:22.190953 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:22.190925    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84ffed29_a741_4bad_a0c1_16e8f4596acc.slice/crio-e64961f6d5bf90cf39a7a021621ce1d9cabb4b32b68e0e5971e873f5dadd4a7c WatchSource:0}: Error finding container e64961f6d5bf90cf39a7a021621ce1d9cabb4b32b68e0e5971e873f5dadd4a7c: Status 404 returned error can't find the container with id e64961f6d5bf90cf39a7a021621ce1d9cabb4b32b68e0e5971e873f5dadd4a7c
Jul 20 16:46:22.223794 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.223763    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn"]
Jul 20 16:46:22.227219 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:22.227190    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8edb7145_cd73_4a73_9383_5d229747f573.slice/crio-78a65b77e50768eeed6fddd28cf92c77b0a2a8bcd20f97aa6cfba8ec9a03f297 WatchSource:0}: Error finding container 78a65b77e50768eeed6fddd28cf92c77b0a2a8bcd20f97aa6cfba8ec9a03f297: Status 404 returned error can't find the container with id 78a65b77e50768eeed6fddd28cf92c77b0a2a8bcd20f97aa6cfba8ec9a03f297
Jul 20 16:46:22.241702 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.241676    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f6f846bcf-jgw5v"]
Jul 20 16:46:22.244177 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.244154    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:22.244849 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:22.244821    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c9724d1_ce8b_46de_b7fb_851f8aba5b98.slice/crio-abf1481c1920914ff752fe817ce702f9ead7df91e30bc34aeec342991c9c7c42 WatchSource:0}: Error finding container abf1481c1920914ff752fe817ce702f9ead7df91e30bc34aeec342991c9c7c42: Status 404 returned error can't find the container with id abf1481c1920914ff752fe817ce702f9ead7df91e30bc34aeec342991c9c7c42
Jul 20 16:46:22.245040 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.245014    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn" event={"ID":"8edb7145-cd73-4a73-9383-5d229747f573","Type":"ContainerStarted","Data":"78a65b77e50768eeed6fddd28cf92c77b0a2a8bcd20f97aa6cfba8ec9a03f297"}
Jul 20 16:46:22.246482 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.246456    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv" event={"ID":"148b5994-e9e4-40d5-8192-2382f30f57e0","Type":"ContainerStarted","Data":"bf7dc0eca6a557a8ed58bec3e787e5882effaa3b4257810756da6e3ef65a2ec7"}
Jul 20 16:46:22.247845 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.247824    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl" event={"ID":"84ffed29-a741-4bad-a0c1-16e8f4596acc","Type":"ContainerStarted","Data":"e64961f6d5bf90cf39a7a021621ce1d9cabb4b32b68e0e5971e873f5dadd4a7c"}
Jul 20 16:46:22.249121 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.249102    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh" event={"ID":"c63ca7b6-b603-474b-bc8a-66d0d0fba0f9","Type":"ContainerStarted","Data":"e5838310afc37607ac561fdee51742f7072b28c7313fef3fd33178c6ec56b9c8"}
Jul 20 16:46:22.438193 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.438151    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:22.614666 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.614637    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:22.715078 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.714943    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:22.815071 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:22.815028    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:23.077384 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:23.077286    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:23.137849 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:23.137797    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:23.210174 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:23.210097    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:23.255763 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:23.255725    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v" event={"ID":"7c9724d1-ce8b-46de-b7fb-851f8aba5b98","Type":"ContainerStarted","Data":"d2bc53429bd57aeeeb1af557807158656e2cd77c167923b715d84826c55a0a37"}
Jul 20 16:46:23.256228 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:23.255792    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v" event={"ID":"7c9724d1-ce8b-46de-b7fb-851f8aba5b98","Type":"ContainerStarted","Data":"abf1481c1920914ff752fe817ce702f9ead7df91e30bc34aeec342991c9c7c42"}
Jul 20 16:46:23.256811 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:23.256593    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:23.268367 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:23.268230    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:23.280624 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:23.279718    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v" podStartSLOduration=2.279700323 podStartE2EDuration="2.279700323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:23.278097825 +0000 UTC m=+62.825953892" watchObservedRunningTime="2026-07-20 16:46:23.279700323 +0000 UTC m=+62.827556367"
Jul 20 16:46:23.594545 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:23.594479    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:25.270529 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.270486    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:25.371140 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.371105    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:25.603477 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.603400    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtcj7\" (UniqueName: \"kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7\") pod \"network-check-target-ql4vb\" (UID: \"ef355774-d5d7-4b50-9643-9ba08afd080d\") " pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:46:25.606475 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.606449    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 16:46:25.616012 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.615978    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 16:46:25.627021 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.626996    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtcj7\" (UniqueName: \"kubernetes.io/projected/ef355774-d5d7-4b50-9643-9ba08afd080d-kube-api-access-qtcj7\") pod \"network-check-target-ql4vb\" (UID: \"ef355774-d5d7-4b50-9643-9ba08afd080d\") " pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:46:25.704432 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.704396    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:46:25.704593 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.704475    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs\") pod \"network-metrics-daemon-m28fr\" (UID: \"87e02cfa-5d5b-4020-ac07-c4cf81f3e099\") " pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:46:25.707718 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.707695    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 16:46:25.707874 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.707743    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 16:46:25.707874 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.707789    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9vk4l\""
Jul 20 16:46:25.714854 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.714831    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:46:25.717778 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.717756    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1e4a352-901a-4050-b0c4-5e63ec823420-original-pull-secret\") pod \"global-pull-secret-syncer-ntr6q\" (UID: \"e1e4a352-901a-4050-b0c4-5e63ec823420\") " pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:46:25.717875 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.717787    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87e02cfa-5d5b-4020-ac07-c4cf81f3e099-metrics-certs\") pod \"network-metrics-daemon-m28fr\" (UID: \"87e02cfa-5d5b-4020-ac07-c4cf81f3e099\") " pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:46:25.802382 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.802347    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntr6q"
Jul 20 16:46:25.809832 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.809809    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2sqpl\""
Jul 20 16:46:25.818035 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:25.818010    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m28fr"
Jul 20 16:46:26.225605 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:26.225575    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-m4g87"
Jul 20 16:46:26.400065 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:26.399975    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:26.500798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:26.500706    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:26.514911 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:26.514835    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ntr6q"]
Jul 20 16:46:26.765224 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:26.765149    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:26.820127 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:26.820097    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:27.012970 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:27.012937    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e4a352_901a_4050_b0c4_5e63ec823420.slice/crio-d04dc9bd6e7c06f2d5ac86958b37aa4614fc3af9a943e06320ce31ac38e6ac15 WatchSource:0}: Error finding container d04dc9bd6e7c06f2d5ac86958b37aa4614fc3af9a943e06320ce31ac38e6ac15: Status 404 returned error can't find the container with id d04dc9bd6e7c06f2d5ac86958b37aa4614fc3af9a943e06320ce31ac38e6ac15
Jul 20 16:46:27.163824 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:27.163793    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m28fr"]
Jul 20 16:46:27.181683 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:27.181618    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ql4vb"]
Jul 20 16:46:27.277725 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:27.277642    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ntr6q" event={"ID":"e1e4a352-901a-4050-b0c4-5e63ec823420","Type":"ContainerStarted","Data":"d04dc9bd6e7c06f2d5ac86958b37aa4614fc3af9a943e06320ce31ac38e6ac15"}
Jul 20 16:46:29.661172 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:29.661133    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:29.787100 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:29.787059    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:29.837476 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:29.837436    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87e02cfa_5d5b_4020_ac07_c4cf81f3e099.slice/crio-f5c24e3ff83fd19cb112e2e8ac1e85889ba48ee84675eca8c79080951487bb9c WatchSource:0}: Error finding container f5c24e3ff83fd19cb112e2e8ac1e85889ba48ee84675eca8c79080951487bb9c: Status 404 returned error can't find the container with id f5c24e3ff83fd19cb112e2e8ac1e85889ba48ee84675eca8c79080951487bb9c
Jul 20 16:46:29.838707 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:29.838376    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef355774_d5d7_4b50_9643_9ba08afd080d.slice/crio-98208e03a2d9c06a6ac4efc29fc407105224b4f472fabcbb71fa5db67a78d77f WatchSource:0}: Error finding container 98208e03a2d9c06a6ac4efc29fc407105224b4f472fabcbb71fa5db67a78d77f: Status 404 returned error can't find the container with id 98208e03a2d9c06a6ac4efc29fc407105224b4f472fabcbb71fa5db67a78d77f
Jul 20 16:46:29.891390 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:29.891353    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:30.291521 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.291482    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn" event={"ID":"8edb7145-cd73-4a73-9383-5d229747f573","Type":"ContainerStarted","Data":"f7f4f9abbea58e2d990d73d9479212f259c59eef1d96790fb0e244bd6ec3e350"}
Jul 20 16:46:30.291701 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.291559    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn" event={"ID":"8edb7145-cd73-4a73-9383-5d229747f573","Type":"ContainerStarted","Data":"f10724ee04bc3ef85a541818c53df5792894f35a31eace155956bfe7ecfaede9"}
Jul 20 16:46:30.291701 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.291580    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn" event={"ID":"8edb7145-cd73-4a73-9383-5d229747f573","Type":"ContainerStarted","Data":"39ac02e0440f74b494a7e512547416f93125fccb81e37cadc1de6505b57cba62"}
Jul 20 16:46:30.293506 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.293473    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ql4vb" event={"ID":"ef355774-d5d7-4b50-9643-9ba08afd080d","Type":"ContainerStarted","Data":"98208e03a2d9c06a6ac4efc29fc407105224b4f472fabcbb71fa5db67a78d77f"}
Jul 20 16:46:30.297645 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.297618    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d8b4g" event={"ID":"30638c7b-0d95-4b98-947d-829c24bb32de","Type":"ContainerStarted","Data":"b5f51c8d170bd15ab3a81204eeb28508d5e50809151d4cd6f888795922205e4c"}
Jul 20 16:46:30.299573 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.299541    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv" event={"ID":"148b5994-e9e4-40d5-8192-2382f30f57e0","Type":"ContainerStarted","Data":"7f7307c0423a301e65f3e9e88ce8c0d752477cd181d369093d6c9b073d7d67c5"}
Jul 20 16:46:30.300009 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.299985    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"
Jul 20 16:46:30.302125 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.302055    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv"
Jul 20 16:46:30.302125 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.302090    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m28fr" event={"ID":"87e02cfa-5d5b-4020-ac07-c4cf81f3e099","Type":"ContainerStarted","Data":"f5c24e3ff83fd19cb112e2e8ac1e85889ba48ee84675eca8c79080951487bb9c"}
Jul 20 16:46:30.303845 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.303816    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl" event={"ID":"84ffed29-a741-4bad-a0c1-16e8f4596acc","Type":"ContainerStarted","Data":"18ffc61afa4561f7e76fbab64b10d336333c26a144c49ecb4183d9a7c5e8155f"}
Jul 20 16:46:30.305654 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.305624    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh" event={"ID":"c63ca7b6-b603-474b-bc8a-66d0d0fba0f9","Type":"ContainerStarted","Data":"d6439b6bed4144d8525a96d016e21989fa6e91ecdcc58b4c266564bf4ed20aee"}
Jul 20 16:46:30.312520 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.311851    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-564cc95d8d-g28dn" podStartSLOduration=1.5634127690000001 podStartE2EDuration="9.311834117s" totalImagesPullingTime="7.748421348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:30.309901084 +0000 UTC m=+69.857757151" watchObservedRunningTime="2026-07-20 16:46:30.311834117 +0000 UTC m=+69.859690177"
Jul 20 16:46:30.335359 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.335322    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f59df4798-86tmv" podStartSLOduration=1.533039407 podStartE2EDuration="9.335304835s" totalImagesPullingTime="7.802265428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:30.334478843 +0000 UTC m=+69.882334909" watchObservedRunningTime="2026-07-20 16:46:30.335304835 +0000 UTC m=+69.883160902"
Jul 20 16:46:30.351282 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.351205    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dc95df6b-mdmwl" podStartSLOduration=1.578646552 podStartE2EDuration="9.351188295s" totalImagesPullingTime="7.772541743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:30.350544328 +0000 UTC m=+69.898400393" watchObservedRunningTime="2026-07-20 16:46:30.351188295 +0000 UTC m=+69.899044362"
Jul 20 16:46:30.372546 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.372506    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f4bbh" podStartSLOduration=1.566607941 podStartE2EDuration="9.372470927s" totalImagesPullingTime="7.805862986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:30.371482355 +0000 UTC m=+69.919338420" watchObservedRunningTime="2026-07-20 16:46:30.372470927 +0000 UTC m=+69.920326992"
Jul 20 16:46:30.498914 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.498784    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:30.554899 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.554861    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:30.623232 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:30.623158    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:31.313316 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:31.313280    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m28fr" event={"ID":"87e02cfa-5d5b-4020-ac07-c4cf81f3e099","Type":"ContainerStarted","Data":"227d888874b6bc77137745edc810c58eae57c8e7eaede42aaa7d5911669f8648"}
Jul 20 16:46:31.572729 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:31.572639    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:31.673169 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:31.673131    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:31.673956 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:31.673667    2568 patch_prober.go:28] interesting pod/image-registry-557b8b5bcc-9tqwn container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 20 16:46:31.673956 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:31.673723    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn" podUID="4085a903-e1c6-4f41-9e07-66ea43060692" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 16:46:32.322569 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:32.322467    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m28fr" event={"ID":"87e02cfa-5d5b-4020-ac07-c4cf81f3e099","Type":"ContainerStarted","Data":"1e4c19b39b65a864974a1bfcee098abd0496f0c5b0a41a6e931c81fc49ea1f18"}
Jul 20 16:46:32.460719 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:32.460684    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:32.526590 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:32.526551    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:32.716778 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:32.716746    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:32.784301 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:32.784267    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:33.032585 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:33.032490    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:33.088251 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:33.088188    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:35.348636 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:35.348603    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:35.404147 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:35.404113    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:36.016628 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:36.016593    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:36.112985 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:36.112950    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:36.847863 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:36.847833    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:36.947664 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:36.947606    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:37.118575 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:37.118498    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:37.217121 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:37.217089    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:37.342670 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:37.342636    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ql4vb" event={"ID":"ef355774-d5d7-4b50-9643-9ba08afd080d","Type":"ContainerStarted","Data":"0ce54b6fe66ca06300af2b23d73118afc53b7f8c7aa5ec2cc2c3bfb0f7385369"}
Jul 20 16:46:37.342833 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:37.342798    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:46:37.344804 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:37.344778    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d8b4g" event={"ID":"30638c7b-0d95-4b98-947d-829c24bb32de","Type":"ContainerStarted","Data":"77960cc3ca8ed1f5bcba26df4ad1c01f59f0428775180ca6a667e16b19c3b427"}
Jul 20 16:46:37.346206 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:37.346178    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ntr6q" event={"ID":"e1e4a352-901a-4050-b0c4-5e63ec823420","Type":"ContainerStarted","Data":"6fb031246e147e96a386fecb5941b7a24cd4c4b87f9ec66d401105281ff49d31"}
Jul 20 16:46:37.359779 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:37.359739    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-m28fr" podStartSLOduration=76.119248514 podStartE2EDuration="1m17.359726794s" totalImagesPullingTime="1.24047828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:32.338754382 +0000 UTC m=+71.886610449" watchObservedRunningTime="2026-07-20 16:46:37.359726794 +0000 UTC m=+76.907582860"
Jul 20 16:46:37.359870 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:37.359840    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ql4vb" podStartSLOduration=70.436160669 podStartE2EDuration="1m17.359834697s" totalImagesPullingTime="6.923674028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:37.358589182 +0000 UTC m=+76.906445252" watchObservedRunningTime="2026-07-20 16:46:37.359834697 +0000 UTC m=+76.907690762"
Jul 20 16:46:37.377411 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:37.377373    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-d8b4g" podStartSLOduration=17.929431134 podStartE2EDuration="39.377360563s" totalImagesPullingTime="21.447929429s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:37.376999898 +0000 UTC m=+76.924855967" watchObservedRunningTime="2026-07-20 16:46:37.377360563 +0000 UTC m=+76.925216630"
Jul 20 16:46:37.391928 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:37.391894    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ntr6q" podStartSLOduration=66.55587539 podStartE2EDuration="1m16.391885541s" totalImagesPullingTime="9.836010151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:37.391871687 +0000 UTC m=+76.939727750" watchObservedRunningTime="2026-07-20 16:46:37.391885541 +0000 UTC m=+76.939741606"
Jul 20 16:46:38.820898 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:38.820859    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:38.921763 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:38.921730    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:39.022354 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:39.022317    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:41.202048 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:41.201995    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:41.669590 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:41.669562    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:46:41.802060 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:41.802027    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:41.855917 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:41.855886    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:42.023737 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:42.023647    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:42.123914 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:42.123878    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:42.921204 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:42.921173    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:43.222293 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:43.222193    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:43.279134 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:43.279106    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:44.139590 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:44.139551    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:44.240180 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:44.240150    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:44.303994 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:44.303960    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:44.945221 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:44.945190    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:45.118379 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.118350    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:45.217762 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.217682    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:45.262740 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.262707    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q7946"]
Jul 20 16:46:45.267469 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.267448    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.270558 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.270533    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 20 16:46:45.270558 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.270559    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2tw6m\""
Jul 20 16:46:45.270747 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.270540    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 20 16:46:45.271709 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.271456    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 20 16:46:45.271709 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.271500    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 20 16:46:45.271709 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.271505    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 20 16:46:45.271709 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.271460    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 20 16:46:45.272200 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.272173    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f6f846bcf-jgw5v"
Jul 20 16:46:45.315933 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.315899    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:45.364769 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.364739    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/231208e4-192f-4ca2-bd06-78c636016542-sys\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.364948 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.364780    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfndg\" (UniqueName: \"kubernetes.io/projected/231208e4-192f-4ca2-bd06-78c636016542-kube-api-access-cfndg\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.364948 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.364807    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/231208e4-192f-4ca2-bd06-78c636016542-metrics-client-ca\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.365071 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.364951    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/231208e4-192f-4ca2-bd06-78c636016542-root\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.365071 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.364988    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-tls\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.365071 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.365050    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-wtmp\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.365214 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.365077    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.365214 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.365134    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-accelerators-collector-config\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.365305 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.365229    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-textfile\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.400476 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.400441    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:45.466179 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466141    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-textfile\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.466358 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466200    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/231208e4-192f-4ca2-bd06-78c636016542-sys\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.466358 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466223    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfndg\" (UniqueName: \"kubernetes.io/projected/231208e4-192f-4ca2-bd06-78c636016542-kube-api-access-cfndg\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.466358 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466265    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/231208e4-192f-4ca2-bd06-78c636016542-metrics-client-ca\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.466358 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466289    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/231208e4-192f-4ca2-bd06-78c636016542-root\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.466358 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466313    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-tls\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.466358 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466347    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-wtmp\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.466596 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466349    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/231208e4-192f-4ca2-bd06-78c636016542-sys\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.466596 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466368    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.466596 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466415    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-accelerators-collector-config\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.466596 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466580    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-textfile\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.466783 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466655    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/231208e4-192f-4ca2-bd06-78c636016542-root\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.466848 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466825    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/231208e4-192f-4ca2-bd06-78c636016542-metrics-client-ca\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.466900 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.466842    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-wtmp\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.467047 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.467027    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-accelerators-collector-config\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.468973 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.468909    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.469290 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.469275    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/231208e4-192f-4ca2-bd06-78c636016542-node-exporter-tls\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.488806 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.488780    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfndg\" (UniqueName: \"kubernetes.io/projected/231208e4-192f-4ca2-bd06-78c636016542-kube-api-access-cfndg\") pod \"node-exporter-q7946\" (UID: \"231208e4-192f-4ca2-bd06-78c636016542\") " pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.513991 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.513957    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:45.579024 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.578991    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q7946"
Jul 20 16:46:45.603633 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:46:45.603595    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod231208e4_192f_4ca2_bd06_78c636016542.slice/crio-e7bbda4a435a037c7ebd078dbf7eab644e7995796ced95bfb826fa4d571af33a WatchSource:0}: Error finding container e7bbda4a435a037c7ebd078dbf7eab644e7995796ced95bfb826fa4d571af33a: Status 404 returned error can't find the container with id e7bbda4a435a037c7ebd078dbf7eab644e7995796ced95bfb826fa4d571af33a
Jul 20 16:46:45.612727 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.612699    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:45.711950 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.711921    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:45.812442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.812360    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:45.913098 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:45.913066    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:46.199034 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:46.198990    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:46.277167 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:46.277121    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:46.383740 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:46.383705    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q7946" event={"ID":"231208e4-192f-4ca2-bd06-78c636016542","Type":"ContainerStarted","Data":"e7bbda4a435a037c7ebd078dbf7eab644e7995796ced95bfb826fa4d571af33a"}
Jul 20 16:46:46.399838 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:46.399802    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:46.503826 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:46.503744    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:46.690819 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:46.690511    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn" podUID="4085a903-e1c6-4f41-9e07-66ea43060692" containerName="registry" containerID="cri-o://60f3bc00a47a7b4b3cfc1fb43e300a4d949753f25ab9dc6bec37c9f4acd5b1e9" gracePeriod=30
Jul 20 16:46:46.937789 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:46.937770    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:46:47.079343 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.079223    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmr9m\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-kube-api-access-jmr9m\") pod \"4085a903-e1c6-4f41-9e07-66ea43060692\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") "
Jul 20 16:46:47.079343 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.079290    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4085a903-e1c6-4f41-9e07-66ea43060692-installation-pull-secrets\") pod \"4085a903-e1c6-4f41-9e07-66ea43060692\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") "
Jul 20 16:46:47.079343 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.079309    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4085a903-e1c6-4f41-9e07-66ea43060692-ca-trust-extracted\") pod \"4085a903-e1c6-4f41-9e07-66ea43060692\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") "
Jul 20 16:46:47.079343 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.079338    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-bound-sa-token\") pod \"4085a903-e1c6-4f41-9e07-66ea43060692\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") "
Jul 20 16:46:47.079671 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.079366    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls\") pod \"4085a903-e1c6-4f41-9e07-66ea43060692\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") "
Jul 20 16:46:47.079671 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.079421    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4085a903-e1c6-4f41-9e07-66ea43060692-registry-certificates\") pod \"4085a903-e1c6-4f41-9e07-66ea43060692\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") "
Jul 20 16:46:47.079671 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.079461    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4085a903-e1c6-4f41-9e07-66ea43060692-image-registry-private-configuration\") pod \"4085a903-e1c6-4f41-9e07-66ea43060692\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") "
Jul 20 16:46:47.079671 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.079510    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4085a903-e1c6-4f41-9e07-66ea43060692-trusted-ca\") pod \"4085a903-e1c6-4f41-9e07-66ea43060692\" (UID: \"4085a903-e1c6-4f41-9e07-66ea43060692\") "
Jul 20 16:46:47.080216 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.080139    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4085a903-e1c6-4f41-9e07-66ea43060692-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4085a903-e1c6-4f41-9e07-66ea43060692" (UID: "4085a903-e1c6-4f41-9e07-66ea43060692"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:46:47.080216 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.080146    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4085a903-e1c6-4f41-9e07-66ea43060692-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4085a903-e1c6-4f41-9e07-66ea43060692" (UID: "4085a903-e1c6-4f41-9e07-66ea43060692"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:46:47.082319 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.082184    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4085a903-e1c6-4f41-9e07-66ea43060692-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4085a903-e1c6-4f41-9e07-66ea43060692" (UID: "4085a903-e1c6-4f41-9e07-66ea43060692"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:46:47.082319 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.082300    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-kube-api-access-jmr9m" (OuterVolumeSpecName: "kube-api-access-jmr9m") pod "4085a903-e1c6-4f41-9e07-66ea43060692" (UID: "4085a903-e1c6-4f41-9e07-66ea43060692"). InnerVolumeSpecName "kube-api-access-jmr9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:46:47.082319 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.082311    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4085a903-e1c6-4f41-9e07-66ea43060692-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4085a903-e1c6-4f41-9e07-66ea43060692" (UID: "4085a903-e1c6-4f41-9e07-66ea43060692"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:46:47.082502 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.082383    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4085a903-e1c6-4f41-9e07-66ea43060692" (UID: "4085a903-e1c6-4f41-9e07-66ea43060692"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:46:47.082502 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.082452    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4085a903-e1c6-4f41-9e07-66ea43060692" (UID: "4085a903-e1c6-4f41-9e07-66ea43060692"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:46:47.089302 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.089273    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4085a903-e1c6-4f41-9e07-66ea43060692-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4085a903-e1c6-4f41-9e07-66ea43060692" (UID: "4085a903-e1c6-4f41-9e07-66ea43060692"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:46:47.180791 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.180758    2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4085a903-e1c6-4f41-9e07-66ea43060692-registry-certificates\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:46:47.180791 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.180785    2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4085a903-e1c6-4f41-9e07-66ea43060692-image-registry-private-configuration\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:46:47.180791 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.180796    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4085a903-e1c6-4f41-9e07-66ea43060692-trusted-ca\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:46:47.181006 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.180804    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmr9m\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-kube-api-access-jmr9m\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:46:47.181006 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.180813    2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4085a903-e1c6-4f41-9e07-66ea43060692-installation-pull-secrets\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:46:47.181006 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.180821    2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4085a903-e1c6-4f41-9e07-66ea43060692-ca-trust-extracted\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:46:47.181006 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.180830    2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-bound-sa-token\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:46:47.181006 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.180838    2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4085a903-e1c6-4f41-9e07-66ea43060692-registry-tls\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:46:47.388186 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.388151    2568 generic.go:356] "Generic (PLEG): container finished" podID="4085a903-e1c6-4f41-9e07-66ea43060692" containerID="60f3bc00a47a7b4b3cfc1fb43e300a4d949753f25ab9dc6bec37c9f4acd5b1e9" exitCode=0
Jul 20 16:46:47.388649 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.388199    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn"
Jul 20 16:46:47.388649 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.388258    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn" event={"ID":"4085a903-e1c6-4f41-9e07-66ea43060692","Type":"ContainerDied","Data":"60f3bc00a47a7b4b3cfc1fb43e300a4d949753f25ab9dc6bec37c9f4acd5b1e9"}
Jul 20 16:46:47.388649 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.388321    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557b8b5bcc-9tqwn" event={"ID":"4085a903-e1c6-4f41-9e07-66ea43060692","Type":"ContainerDied","Data":"eb50dadc25c20c9c9642d78a0dd7831ac5dee5912f297614136a883e9fb8d0fd"}
Jul 20 16:46:47.388649 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.388350    2568 scope.go:117] "RemoveContainer" containerID="60f3bc00a47a7b4b3cfc1fb43e300a4d949753f25ab9dc6bec37c9f4acd5b1e9"
Jul 20 16:46:47.390309 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.390282    2568 generic.go:356] "Generic (PLEG): container finished" podID="231208e4-192f-4ca2-bd06-78c636016542" containerID="addb49aac7f78fc49c694a553ad8927f4c68c5191fe9ad1764a4ca933dceaa15" exitCode=0
Jul 20 16:46:47.390411 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.390341    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q7946" event={"ID":"231208e4-192f-4ca2-bd06-78c636016542","Type":"ContainerDied","Data":"addb49aac7f78fc49c694a553ad8927f4c68c5191fe9ad1764a4ca933dceaa15"}
Jul 20 16:46:47.397580 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.397559    2568 scope.go:117] "RemoveContainer" containerID="60f3bc00a47a7b4b3cfc1fb43e300a4d949753f25ab9dc6bec37c9f4acd5b1e9"
Jul 20 16:46:47.397846 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:46:47.397827    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60f3bc00a47a7b4b3cfc1fb43e300a4d949753f25ab9dc6bec37c9f4acd5b1e9\": container with ID starting with 60f3bc00a47a7b4b3cfc1fb43e300a4d949753f25ab9dc6bec37c9f4acd5b1e9 not found: ID does not exist" containerID="60f3bc00a47a7b4b3cfc1fb43e300a4d949753f25ab9dc6bec37c9f4acd5b1e9"
Jul 20 16:46:47.397915 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.397876    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60f3bc00a47a7b4b3cfc1fb43e300a4d949753f25ab9dc6bec37c9f4acd5b1e9"} err="failed to get container status \"60f3bc00a47a7b4b3cfc1fb43e300a4d949753f25ab9dc6bec37c9f4acd5b1e9\": rpc error: code = NotFound desc = could not find container \"60f3bc00a47a7b4b3cfc1fb43e300a4d949753f25ab9dc6bec37c9f4acd5b1e9\": container with ID starting with 60f3bc00a47a7b4b3cfc1fb43e300a4d949753f25ab9dc6bec37c9f4acd5b1e9 not found: ID does not exist"
Jul 20 16:46:47.422216 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.422193    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-557b8b5bcc-9tqwn"]
Jul 20 16:46:47.425653 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:47.425634    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-557b8b5bcc-9tqwn"]
Jul 20 16:46:48.131376 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:48.131342    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:48.211252 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:48.211197    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:48.396368 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:48.396271    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q7946" event={"ID":"231208e4-192f-4ca2-bd06-78c636016542","Type":"ContainerStarted","Data":"f22a6c6a0cf811bb05cae89809df515fa4585b45323d7716bc90384c7356da20"}
Jul 20 16:46:48.396368 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:48.396336    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q7946" event={"ID":"231208e4-192f-4ca2-bd06-78c636016542","Type":"ContainerStarted","Data":"e47fa7857982b448c24b593e060a331e2247a0b38c7b8e456ce952d13fdfcba8"}
Jul 20 16:46:48.420561 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:48.420495    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q7946" podStartSLOduration=2.1720080250000002 podStartE2EDuration="3.420473731s" totalImagesPullingTime="1.172827606s" totalInitContainerRuntime="75.6381ms" isStatefulPod=true podCreationTimestamp="2026-07-20 16:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:48.417538782 +0000 UTC m=+87.965394852" watchObservedRunningTime="2026-07-20 16:46:48.420473731 +0000 UTC m=+87.968329798"
Jul 20 16:46:48.445522 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:48.445460    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:48.545895 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:48.545854    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:48.894680 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:48.894649    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4085a903-e1c6-4f41-9e07-66ea43060692" path="/var/lib/kubelet/pods/4085a903-e1c6-4f41-9e07-66ea43060692/volumes"
Jul 20 16:46:50.422933 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:50.422894    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:50.504540 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:50.504490    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:51.499729 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:51.499691    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:51.849365 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:51.849291    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:51.944977 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:51.944947    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:53.126358 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:53.126328    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:53.227717 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:53.227684    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:54.249447 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:54.249417    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:54.350483 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:54.350449    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:58.050443 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:58.050410    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:46:58.106672 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:46:58.106645    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:47:03.210197 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:47:03.210167    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:47:03.263946 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:47:03.263914    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:47:06.839846 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:47:06.839806    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:47:06.899913 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:47:06.899887    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:47:08.352216 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:47:08.352184    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ql4vb"
Jul 20 16:47:12.461978 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:47:12.461950    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:47:12.563030 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:47:12.562990    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:47:12.617957 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:47:12.617916    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:47:21.842422 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:47:21.842385    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:47:21.902706 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:47:21.902669    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:47:25.823423 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:47:25.823386    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:47:25.878655 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:47:25.878620    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:01.309060 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:01.309026    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:01.387203 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:01.387174    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:13.216297 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:13.216201    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:13.285118 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:13.285051    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:21.860224 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:21.860186    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:21.955062 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:21.955026    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:22.200229 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:22.200200    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:22.259367 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:22.259318    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:22.355625 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:22.355592    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:23.118835 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:23.118806    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:23.174167 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:23.174141    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:32.818254 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:32.818211    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:32.919560 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:32.919528    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:32.985003 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:32.984970    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:49.796911 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:49.796868    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:49.860298 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:48:49.860265    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:04.590843 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:04.590799    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:04.833008 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:04.832969    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:17.951255 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:17.951201    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:32.085870 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:32.085839    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:32.945868 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:32.945833    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:35.903711 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:35.903678    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:36.003777 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:36.003742    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:37.477956 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:37.477924    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:37.606503 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:37.606467    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:38.278870 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:38.278831    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:38.497748 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:38.497686    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:40.428139 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:40.428099    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:40.811077 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:40.811000    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:42.684372 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:42.684282    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:42.788982 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:42.788947    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:48.457367 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:48.457330    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:54.123050 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:54.123011    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:54.238643 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:49:54.238611    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:00.814527 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:00.814497    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:00.915169 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:00.915136    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:01.132627 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:01.132596    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:01.232127 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:01.232087    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:03.032191 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:03.032158    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:15.210972 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:15.210943    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:16.592161 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.592131    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"]
Jul 20 16:50:16.592531 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.592393    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4085a903-e1c6-4f41-9e07-66ea43060692" containerName="registry"
Jul 20 16:50:16.592531 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.592404    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4085a903-e1c6-4f41-9e07-66ea43060692" containerName="registry"
Jul 20 16:50:16.592531 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.592448    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4085a903-e1c6-4f41-9e07-66ea43060692" containerName="registry"
Jul 20 16:50:16.595217 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.595197    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.598746 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.598720    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 20 16:50:16.599522 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.599316    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 20 16:50:16.599658 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.599636    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 20 16:50:16.599860 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.599835    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zwkbp\""
Jul 20 16:50:16.599972 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.599906    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 20 16:50:16.630990 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.630966    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"]
Jul 20 16:50:16.790503 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.790471    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-manifests\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.790503 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.790505    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-tmp\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.790750 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.790524    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.790750 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.790542    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h76w\" (UniqueName: \"kubernetes.io/projected/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-kube-api-access-7h76w\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.790750 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.790645    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-webhook-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.891398 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.891370    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-manifests\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.891557 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.891412    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-tmp\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.891557 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.891445    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.891557 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.891473    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h76w\" (UniqueName: \"kubernetes.io/projected/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-kube-api-access-7h76w\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.891557 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.891515    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-webhook-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.891829 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.891795    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-manifests\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.893929 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.893896    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-tmp\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.894036 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.894017    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-webhook-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.894095 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.894068    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.902331 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.902308    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h76w\" (UniqueName: \"kubernetes.io/projected/af5d97d7-26f3-49f6-b2b2-44ae102a5ab4-kube-api-access-7h76w\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-f9gv6\" (UID: \"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:16.908175 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:16.908149    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:17.090103 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:50:17.090072    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf5d97d7_26f3_49f6_b2b2_44ae102a5ab4.slice/crio-d7b14f178ab000809113ae9f3ffc51a70b2e7d79317c0f3a1278652daed1d0c3 WatchSource:0}: Error finding container d7b14f178ab000809113ae9f3ffc51a70b2e7d79317c0f3a1278652daed1d0c3: Status 404 returned error can't find the container with id d7b14f178ab000809113ae9f3ffc51a70b2e7d79317c0f3a1278652daed1d0c3
Jul 20 16:50:17.092466 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:17.092417    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"]
Jul 20 16:50:17.181110 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:17.181043    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:17.329649 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:17.329615    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:17.500755 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:17.500671    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:18.079728 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:18.079692    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6" event={"ID":"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4","Type":"ContainerStarted","Data":"d7b14f178ab000809113ae9f3ffc51a70b2e7d79317c0f3a1278652daed1d0c3"}
Jul 20 16:50:20.777930 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:20.777903    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 16:50:20.781690 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:20.781670    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 16:50:20.782540 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:20.782518    2568 kubelet.go:1648] "Image garbage collection succeeded"
Jul 20 16:50:21.093556 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:21.093378    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6" event={"ID":"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4","Type":"ContainerStarted","Data":"b267801fee22c33a1b1886d6e23dcf68084ef52e232c51ccc55d94a66600bc9e"}
Jul 20 16:50:22.098477 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:22.098448    2568 generic.go:356] "Generic (PLEG): container finished" podID="af5d97d7-26f3-49f6-b2b2-44ae102a5ab4" containerID="b267801fee22c33a1b1886d6e23dcf68084ef52e232c51ccc55d94a66600bc9e" exitCode=0
Jul 20 16:50:22.098849 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:22.098498    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6" event={"ID":"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4","Type":"ContainerDied","Data":"b267801fee22c33a1b1886d6e23dcf68084ef52e232c51ccc55d94a66600bc9e"}
Jul 20 16:50:22.098962 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:22.098946    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 16:50:23.103198 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:23.103163    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6" event={"ID":"af5d97d7-26f3-49f6-b2b2-44ae102a5ab4","Type":"ContainerStarted","Data":"2e0c8d066844686d899a03328a5f5642295f779b8d8e5e2974717ba483873f4e"}
Jul 20 16:50:23.103659 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:23.103278    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:23.135042 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:23.134999    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6" podStartSLOduration=2.7189779229999997 podStartE2EDuration="7.134979749s" totalImagesPullingTime="3.367480102s" totalInitContainerRuntime="1.048521724s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:23.133121258 +0000 UTC m=+302.680977323" watchObservedRunningTime="2026-07-20 16:50:23.134979749 +0000 UTC m=+302.682835812"
Jul 20 16:50:27.441096 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:27.441064    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:27.652058 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:27.652015    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:34.109730 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:34.109704    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-f9gv6"
Jul 20 16:50:36.070939 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:36.070907    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:36.292792 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:36.292757    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:36.942876 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:36.942842    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:37.051075 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.051044    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"]
Jul 20 16:50:37.064224 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.064200    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"
Jul 20 16:50:37.069558 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.069535    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 20 16:50:37.069812 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.069796    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 20 16:50:37.069896 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.069866    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 20 16:50:37.069962 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.069917    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 20 16:50:37.089270 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.089225    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"]
Jul 20 16:50:37.135833 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.135799    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgkrc\" (UniqueName: \"kubernetes.io/projected/b6e7bb43-215a-4c22-9621-b4ca25e1c223-kube-api-access-tgkrc\") pod \"kube-auth-proxy-769f9dd78c-9gdz7\" (UID: \"b6e7bb43-215a-4c22-9621-b4ca25e1c223\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"
Jul 20 16:50:37.136011 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.135844    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6e7bb43-215a-4c22-9621-b4ca25e1c223-tmp\") pod \"kube-auth-proxy-769f9dd78c-9gdz7\" (UID: \"b6e7bb43-215a-4c22-9621-b4ca25e1c223\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"
Jul 20 16:50:37.136011 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.135931    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6e7bb43-215a-4c22-9621-b4ca25e1c223-tls-certs\") pod \"kube-auth-proxy-769f9dd78c-9gdz7\" (UID: \"b6e7bb43-215a-4c22-9621-b4ca25e1c223\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"
Jul 20 16:50:37.236628 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.236532    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6e7bb43-215a-4c22-9621-b4ca25e1c223-tls-certs\") pod \"kube-auth-proxy-769f9dd78c-9gdz7\" (UID: \"b6e7bb43-215a-4c22-9621-b4ca25e1c223\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"
Jul 20 16:50:37.236628 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.236606    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgkrc\" (UniqueName: \"kubernetes.io/projected/b6e7bb43-215a-4c22-9621-b4ca25e1c223-kube-api-access-tgkrc\") pod \"kube-auth-proxy-769f9dd78c-9gdz7\" (UID: \"b6e7bb43-215a-4c22-9621-b4ca25e1c223\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"
Jul 20 16:50:37.236862 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.236660    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6e7bb43-215a-4c22-9621-b4ca25e1c223-tmp\") pod \"kube-auth-proxy-769f9dd78c-9gdz7\" (UID: \"b6e7bb43-215a-4c22-9621-b4ca25e1c223\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"
Jul 20 16:50:37.239006 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.238979    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6e7bb43-215a-4c22-9621-b4ca25e1c223-tmp\") pod \"kube-auth-proxy-769f9dd78c-9gdz7\" (UID: \"b6e7bb43-215a-4c22-9621-b4ca25e1c223\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"
Jul 20 16:50:37.239214 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.239196    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6e7bb43-215a-4c22-9621-b4ca25e1c223-tls-certs\") pod \"kube-auth-proxy-769f9dd78c-9gdz7\" (UID: \"b6e7bb43-215a-4c22-9621-b4ca25e1c223\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"
Jul 20 16:50:37.244446 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.244423    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:37.251667 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.251644    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgkrc\" (UniqueName: \"kubernetes.io/projected/b6e7bb43-215a-4c22-9621-b4ca25e1c223-kube-api-access-tgkrc\") pod \"kube-auth-proxy-769f9dd78c-9gdz7\" (UID: \"b6e7bb43-215a-4c22-9621-b4ca25e1c223\") " pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"
Jul 20 16:50:37.373724 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.373689    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"
Jul 20 16:50:37.448328 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.448302    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7l9vf"]
Jul 20 16:50:37.452169 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.451769    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 16:50:37.455465 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.455445    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-mxzgq\""
Jul 20 16:50:37.455859 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.455843    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 20 16:50:37.490543 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.490470    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7l9vf"]
Jul 20 16:50:37.539044 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.539009    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a98944f-4afb-4716-978a-58b04ff33c50-cert\") pod \"odh-model-controller-7998c67dc4-7l9vf\" (UID: \"9a98944f-4afb-4716-978a-58b04ff33c50\") " pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 16:50:37.539194 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.539062    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csjvw\" (UniqueName: \"kubernetes.io/projected/9a98944f-4afb-4716-978a-58b04ff33c50-kube-api-access-csjvw\") pod \"odh-model-controller-7998c67dc4-7l9vf\" (UID: \"9a98944f-4afb-4716-978a-58b04ff33c50\") " pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 16:50:37.554401 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.554376    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7"]
Jul 20 16:50:37.639931 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.639898    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csjvw\" (UniqueName: \"kubernetes.io/projected/9a98944f-4afb-4716-978a-58b04ff33c50-kube-api-access-csjvw\") pod \"odh-model-controller-7998c67dc4-7l9vf\" (UID: \"9a98944f-4afb-4716-978a-58b04ff33c50\") " pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 16:50:37.640090 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.639968    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a98944f-4afb-4716-978a-58b04ff33c50-cert\") pod \"odh-model-controller-7998c67dc4-7l9vf\" (UID: \"9a98944f-4afb-4716-978a-58b04ff33c50\") " pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 16:50:37.640143 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:50:37.640104    2568 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 20 16:50:37.640194 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:50:37.640188    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a98944f-4afb-4716-978a-58b04ff33c50-cert podName:9a98944f-4afb-4716-978a-58b04ff33c50 nodeName:}" failed. No retries permitted until 2026-07-20 16:50:38.140166903 +0000 UTC m=+317.688022953 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9a98944f-4afb-4716-978a-58b04ff33c50-cert") pod "odh-model-controller-7998c67dc4-7l9vf" (UID: "9a98944f-4afb-4716-978a-58b04ff33c50") : secret "odh-model-controller-webhook-cert" not found
Jul 20 16:50:37.649296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.649269    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:37.665684 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.665656    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csjvw\" (UniqueName: \"kubernetes.io/projected/9a98944f-4afb-4716-978a-58b04ff33c50-kube-api-access-csjvw\") pod \"odh-model-controller-7998c67dc4-7l9vf\" (UID: \"9a98944f-4afb-4716-978a-58b04ff33c50\") " pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 16:50:37.803279 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:37.803166    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:38.003462 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:38.003427    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:38.143968 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:38.143938    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a98944f-4afb-4716-978a-58b04ff33c50-cert\") pod \"odh-model-controller-7998c67dc4-7l9vf\" (UID: \"9a98944f-4afb-4716-978a-58b04ff33c50\") " pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 16:50:38.146458 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:38.146441    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a98944f-4afb-4716-978a-58b04ff33c50-cert\") pod \"odh-model-controller-7998c67dc4-7l9vf\" (UID: \"9a98944f-4afb-4716-978a-58b04ff33c50\") " pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 16:50:38.156297 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:38.156273    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7" event={"ID":"b6e7bb43-215a-4c22-9621-b4ca25e1c223","Type":"ContainerStarted","Data":"fc1a74556d50ea69266741fc649d9fe4fa28299be7265896decfc9eb0164bf38"}
Jul 20 16:50:38.368491 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:38.368453    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 16:50:38.444324 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:38.444050    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:38.553173 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:38.553071    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7l9vf"]
Jul 20 16:50:38.556033 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:50:38.555990    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a98944f_4afb_4716_978a_58b04ff33c50.slice/crio-537e4ab995d9ceb3d25c0e809452e11f0eb89ab64202c0fa5bae29b10819e0ac WatchSource:0}: Error finding container 537e4ab995d9ceb3d25c0e809452e11f0eb89ab64202c0fa5bae29b10819e0ac: Status 404 returned error can't find the container with id 537e4ab995d9ceb3d25c0e809452e11f0eb89ab64202c0fa5bae29b10819e0ac
Jul 20 16:50:39.161993 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:39.161959    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf" event={"ID":"9a98944f-4afb-4716-978a-58b04ff33c50","Type":"ContainerStarted","Data":"537e4ab995d9ceb3d25c0e809452e11f0eb89ab64202c0fa5bae29b10819e0ac"}
Jul 20 16:50:39.401201 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:39.401164    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zrjc7"]
Jul 20 16:50:39.404779 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:39.404750    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7"
Jul 20 16:50:39.417356 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:39.416591    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 20 16:50:39.442903 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:39.442870    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zrjc7"]
Jul 20 16:50:39.455253 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:39.455205    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/134f64e8-eb77-4709-9af2-f2081bbf2aaa-tls-certs\") pod \"model-serving-api-54d89bd5fc-zrjc7\" (UID: \"134f64e8-eb77-4709-9af2-f2081bbf2aaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7"
Jul 20 16:50:39.455437 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:39.455357    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fjk6\" (UniqueName: \"kubernetes.io/projected/134f64e8-eb77-4709-9af2-f2081bbf2aaa-kube-api-access-4fjk6\") pod \"model-serving-api-54d89bd5fc-zrjc7\" (UID: \"134f64e8-eb77-4709-9af2-f2081bbf2aaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7"
Jul 20 16:50:39.477487 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:39.477410    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:39.556392 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:39.555955    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fjk6\" (UniqueName: \"kubernetes.io/projected/134f64e8-eb77-4709-9af2-f2081bbf2aaa-kube-api-access-4fjk6\") pod \"model-serving-api-54d89bd5fc-zrjc7\" (UID: \"134f64e8-eb77-4709-9af2-f2081bbf2aaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7"
Jul 20 16:50:39.556392 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:39.556038    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/134f64e8-eb77-4709-9af2-f2081bbf2aaa-tls-certs\") pod \"model-serving-api-54d89bd5fc-zrjc7\" (UID: \"134f64e8-eb77-4709-9af2-f2081bbf2aaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7"
Jul 20 16:50:39.556392 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:50:39.556218    2568 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 20 16:50:39.556392 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:50:39.556307    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/134f64e8-eb77-4709-9af2-f2081bbf2aaa-tls-certs podName:134f64e8-eb77-4709-9af2-f2081bbf2aaa nodeName:}" failed. No retries permitted until 2026-07-20 16:50:40.05628537 +0000 UTC m=+319.604141414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/134f64e8-eb77-4709-9af2-f2081bbf2aaa-tls-certs") pod "model-serving-api-54d89bd5fc-zrjc7" (UID: "134f64e8-eb77-4709-9af2-f2081bbf2aaa") : secret "model-serving-api-tls" not found
Jul 20 16:50:39.585382 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:39.585303    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fjk6\" (UniqueName: \"kubernetes.io/projected/134f64e8-eb77-4709-9af2-f2081bbf2aaa-kube-api-access-4fjk6\") pod \"model-serving-api-54d89bd5fc-zrjc7\" (UID: \"134f64e8-eb77-4709-9af2-f2081bbf2aaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7"
Jul 20 16:50:40.061745 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.061643    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/134f64e8-eb77-4709-9af2-f2081bbf2aaa-tls-certs\") pod \"model-serving-api-54d89bd5fc-zrjc7\" (UID: \"134f64e8-eb77-4709-9af2-f2081bbf2aaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7"
Jul 20 16:50:40.064761 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.064734    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/134f64e8-eb77-4709-9af2-f2081bbf2aaa-tls-certs\") pod \"model-serving-api-54d89bd5fc-zrjc7\" (UID: \"134f64e8-eb77-4709-9af2-f2081bbf2aaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7"
Jul 20 16:50:40.123310 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.123265    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:40.319905 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.319814    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7"
Jul 20 16:50:40.570395 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.570228    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:40.582957 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.582931    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"]
Jul 20 16:50:40.586019 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.585999    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.590725 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.590704    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 20 16:50:40.590725 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.590711    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 20 16:50:40.590725 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.590725    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 16:50:40.590977 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.590707    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 20 16:50:40.600352 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.600328    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-lffvv\""
Jul 20 16:50:40.616048 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.616020    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"]
Jul 20 16:50:40.667941 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.667909    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.668144 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.667959    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-cacerts\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.668144 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.667991    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.668144 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.668058    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26m8j\" (UniqueName: \"kubernetes.io/projected/7c79f856-187b-4c0e-ae1e-c852455c9819-kube-api-access-26m8j\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.668144 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.668122    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c79f856-187b-4c0e-ae1e-c852455c9819-local-certs\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.668399 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.668174    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-token\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.668399 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.668294    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.722995 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.722961    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:40.769308 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.769275    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-cacerts\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.769487 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.769324    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.769550 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.769482    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26m8j\" (UniqueName: \"kubernetes.io/projected/7c79f856-187b-4c0e-ae1e-c852455c9819-kube-api-access-26m8j\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.769607 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.769557    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c79f856-187b-4c0e-ae1e-c852455c9819-local-certs\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.769607 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.769594    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-token\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.770269 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.769769    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.770269 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.769825    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.770269 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.769949    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.772348 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.772323    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-cacerts\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.772568 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.772549    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.772636 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.772585    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c79f856-187b-4c0e-ae1e-c852455c9819-local-certs\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.772768 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.772746    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.779653 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.779603    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26m8j\" (UniqueName: \"kubernetes.io/projected/7c79f856-187b-4c0e-ae1e-c852455c9819-kube-api-access-26m8j\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.779785 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.779675    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-token\") pod \"istiod-openshift-gateway-7574459656-shmkf\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:40.897193 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:40.897155    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:41.718035 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:41.718008    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zrjc7"]
Jul 20 16:50:41.720084 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:50:41.720058    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod134f64e8_eb77_4709_9af2_f2081bbf2aaa.slice/crio-8cee6d48bf681d176fbdd712d6a863dcf230b0c2edafe5c9f738497bb916a1e4 WatchSource:0}: Error finding container 8cee6d48bf681d176fbdd712d6a863dcf230b0c2edafe5c9f738497bb916a1e4: Status 404 returned error can't find the container with id 8cee6d48bf681d176fbdd712d6a863dcf230b0c2edafe5c9f738497bb916a1e4
Jul 20 16:50:41.764708 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:41.764656    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"]
Jul 20 16:50:41.771930 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:50:41.771896    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c79f856_187b_4c0e_ae1e_c852455c9819.slice/crio-15a9200fce158cc3f77b976bb9c61755098e17c285121fff3e112c7fb796473a WatchSource:0}: Error finding container 15a9200fce158cc3f77b976bb9c61755098e17c285121fff3e112c7fb796473a: Status 404 returned error can't find the container with id 15a9200fce158cc3f77b976bb9c61755098e17c285121fff3e112c7fb796473a
Jul 20 16:50:42.176821 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:42.176780    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf" event={"ID":"9a98944f-4afb-4716-978a-58b04ff33c50","Type":"ContainerStarted","Data":"c2a1c522809c89fcc693ac0e4259fd8b83c56b16ca18137fbea008f2d2ebfd88"}
Jul 20 16:50:42.177032 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:42.176893    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 16:50:42.178054 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:42.178027    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf" event={"ID":"7c79f856-187b-4c0e-ae1e-c852455c9819","Type":"ContainerStarted","Data":"15a9200fce158cc3f77b976bb9c61755098e17c285121fff3e112c7fb796473a"}
Jul 20 16:50:42.179008 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:42.178986    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7" event={"ID":"134f64e8-eb77-4709-9af2-f2081bbf2aaa","Type":"ContainerStarted","Data":"8cee6d48bf681d176fbdd712d6a863dcf230b0c2edafe5c9f738497bb916a1e4"}
Jul 20 16:50:42.180197 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:42.180178    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7" event={"ID":"b6e7bb43-215a-4c22-9621-b4ca25e1c223","Type":"ContainerStarted","Data":"7535ba02d2e9f18136fec63fee692f46fc2a828c9c23a7398b3752979bc6a0de"}
Jul 20 16:50:42.236850 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:42.236792    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf" podStartSLOduration=2.18024664 podStartE2EDuration="5.236773036s" totalImagesPullingTime="3.056526396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:42.206965013 +0000 UTC m=+321.754821077" watchObservedRunningTime="2026-07-20 16:50:42.236773036 +0000 UTC m=+321.784629151"
Jul 20 16:50:42.237062 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:42.237029    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-769f9dd78c-9gdz7" podStartSLOduration=2.236331137 podStartE2EDuration="6.237017064s" totalImagesPullingTime="4.000685927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:42.234856049 +0000 UTC m=+321.782712113" watchObservedRunningTime="2026-07-20 16:50:42.237017064 +0000 UTC m=+321.784873131"
Jul 20 16:50:42.899782 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:42.899748    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:43.192273 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:43.192165    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:44.189388 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:44.189348    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7" event={"ID":"134f64e8-eb77-4709-9af2-f2081bbf2aaa","Type":"ContainerStarted","Data":"b39c4f9ec61c5f73d80c5ebbc67f17401df242ec5c2e5bc54186936fe78f374e"}
Jul 20 16:50:44.189746 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:44.189412    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7"
Jul 20 16:50:44.291535 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:44.291484    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7" podStartSLOduration=3.455128463 podStartE2EDuration="5.291466399s" totalImagesPullingTime="1.836337936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:44.289351331 +0000 UTC m=+323.837207397" watchObservedRunningTime="2026-07-20 16:50:44.291466399 +0000 UTC m=+323.839322464"
Jul 20 16:50:45.936439 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:45.936409    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 20 16:50:45.936684 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:45.936481    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 20 16:50:46.199457 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:46.199365    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf" event={"ID":"7c79f856-187b-4c0e-ae1e-c852455c9819","Type":"ContainerStarted","Data":"d1eef268396fc779b5ba482f31e2dbfd5d6e59905102bd1f160cc41d44e9e0af"}
Jul 20 16:50:46.199617 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:46.199547    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:46.201333 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:46.201306    2568 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-shmkf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 16:50:46.201460 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:46.201360    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf" podUID="7c79f856-187b-4c0e-ae1e-c852455c9819" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 16:50:46.719549 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:46.719515    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:47.043406 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:47.043313    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:47.203685 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:47.203650    2568 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-shmkf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 16:50:47.203835 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:47.203715    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf" podUID="7c79f856-187b-4c0e-ae1e-c852455c9819" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 16:50:48.697651 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:48.697618    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:49.046988 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:49.046905    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:49.258285 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:49.258230    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:50.204598 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:50.204572    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:50.272223 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:50.272172    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf" podStartSLOduration=6.109957179 podStartE2EDuration="10.272155033s" totalImagesPullingTime="4.162197854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:46.261773421 +0000 UTC m=+325.809629485" watchObservedRunningTime="2026-07-20 16:50:50.272155033 +0000 UTC m=+329.820011098"
Jul 20 16:50:52.992012 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:52.991950    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:53.185752 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.185729    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 16:50:53.284615 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.284542    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp"]
Jul 20 16:50:53.287830 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.287815    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp"
Jul 20 16:50:53.311313 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.311289    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-5v45c\""
Jul 20 16:50:53.311442 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.311427    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 20 16:50:53.332163 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.332136    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp"]
Jul 20 16:50:53.334494 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.334471    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:53.378719 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.378689    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e43ba35f-7425-48b4-a81c-49dcfb086017-cert\") pod \"kserve-controller-manager-d6f4b65b8-lxdxp\" (UID: \"e43ba35f-7425-48b4-a81c-49dcfb086017\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp"
Jul 20 16:50:53.378719 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.378717    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb866\" (UniqueName: \"kubernetes.io/projected/e43ba35f-7425-48b4-a81c-49dcfb086017-kube-api-access-gb866\") pod \"kserve-controller-manager-d6f4b65b8-lxdxp\" (UID: \"e43ba35f-7425-48b4-a81c-49dcfb086017\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp"
Jul 20 16:50:53.459193 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.459162    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8"]
Jul 20 16:50:53.462318 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.462301    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8"
Jul 20 16:50:53.477063 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.477045    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 20 16:50:53.479389 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.479370    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e43ba35f-7425-48b4-a81c-49dcfb086017-cert\") pod \"kserve-controller-manager-d6f4b65b8-lxdxp\" (UID: \"e43ba35f-7425-48b4-a81c-49dcfb086017\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp"
Jul 20 16:50:53.479479 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.479400    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb866\" (UniqueName: \"kubernetes.io/projected/e43ba35f-7425-48b4-a81c-49dcfb086017-kube-api-access-gb866\") pod \"kserve-controller-manager-d6f4b65b8-lxdxp\" (UID: \"e43ba35f-7425-48b4-a81c-49dcfb086017\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp"
Jul 20 16:50:53.482166 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.482149    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e43ba35f-7425-48b4-a81c-49dcfb086017-cert\") pod \"kserve-controller-manager-d6f4b65b8-lxdxp\" (UID: \"e43ba35f-7425-48b4-a81c-49dcfb086017\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp"
Jul 20 16:50:53.511567 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.511493    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-hk6sw\""
Jul 20 16:50:53.512561 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.512535    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8"]
Jul 20 16:50:53.536815 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.536756    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb866\" (UniqueName: \"kubernetes.io/projected/e43ba35f-7425-48b4-a81c-49dcfb086017-kube-api-access-gb866\") pod \"kserve-controller-manager-d6f4b65b8-lxdxp\" (UID: \"e43ba35f-7425-48b4-a81c-49dcfb086017\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp"
Jul 20 16:50:53.542474 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.542414    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:53.580277 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.580227    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5acb4774-f9cf-4e43-94b1-d990d3033b11-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kd9c8\" (UID: \"5acb4774-f9cf-4e43-94b1-d990d3033b11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8"
Jul 20 16:50:53.580441 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.580290    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phpbs\" (UniqueName: \"kubernetes.io/projected/5acb4774-f9cf-4e43-94b1-d990d3033b11-kube-api-access-phpbs\") pod \"llmisvc-controller-manager-777c6bd7d5-kd9c8\" (UID: \"5acb4774-f9cf-4e43-94b1-d990d3033b11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8"
Jul 20 16:50:53.597213 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.597192    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp"
Jul 20 16:50:53.681571 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.680875    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5acb4774-f9cf-4e43-94b1-d990d3033b11-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kd9c8\" (UID: \"5acb4774-f9cf-4e43-94b1-d990d3033b11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8"
Jul 20 16:50:53.681571 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.680973    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phpbs\" (UniqueName: \"kubernetes.io/projected/5acb4774-f9cf-4e43-94b1-d990d3033b11-kube-api-access-phpbs\") pod \"llmisvc-controller-manager-777c6bd7d5-kd9c8\" (UID: \"5acb4774-f9cf-4e43-94b1-d990d3033b11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8"
Jul 20 16:50:53.684735 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.684696    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5acb4774-f9cf-4e43-94b1-d990d3033b11-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kd9c8\" (UID: \"5acb4774-f9cf-4e43-94b1-d990d3033b11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8"
Jul 20 16:50:53.726173 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.726148    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phpbs\" (UniqueName: \"kubernetes.io/projected/5acb4774-f9cf-4e43-94b1-d990d3033b11-kube-api-access-phpbs\") pod \"llmisvc-controller-manager-777c6bd7d5-kd9c8\" (UID: \"5acb4774-f9cf-4e43-94b1-d990d3033b11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8"
Jul 20 16:50:53.771661 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.771604    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8"
Jul 20 16:50:53.773330 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.773303    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp"]
Jul 20 16:50:53.776182 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:50:53.776161    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode43ba35f_7425_48b4_a81c_49dcfb086017.slice/crio-c29608960db34188f480f86fa3cdaedf54401f59bdfdaa7432efc924515062ea WatchSource:0}: Error finding container c29608960db34188f480f86fa3cdaedf54401f59bdfdaa7432efc924515062ea: Status 404 returned error can't find the container with id c29608960db34188f480f86fa3cdaedf54401f59bdfdaa7432efc924515062ea
Jul 20 16:50:53.915577 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:53.915550    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8"]
Jul 20 16:50:53.918681 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:50:53.918657    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5acb4774_f9cf_4e43_94b1_d990d3033b11.slice/crio-0192067de05973e4af63a7825d86cba10aad0b8e6ce8c52246d56ba1f338666d WatchSource:0}: Error finding container 0192067de05973e4af63a7825d86cba10aad0b8e6ce8c52246d56ba1f338666d: Status 404 returned error can't find the container with id 0192067de05973e4af63a7825d86cba10aad0b8e6ce8c52246d56ba1f338666d
Jul 20 16:50:54.229760 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:54.229728    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8" event={"ID":"5acb4774-f9cf-4e43-94b1-d990d3033b11","Type":"ContainerStarted","Data":"0192067de05973e4af63a7825d86cba10aad0b8e6ce8c52246d56ba1f338666d"}
Jul 20 16:50:54.230856 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:54.230833    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp" event={"ID":"e43ba35f-7425-48b4-a81c-49dcfb086017","Type":"ContainerStarted","Data":"c29608960db34188f480f86fa3cdaedf54401f59bdfdaa7432efc924515062ea"}
Jul 20 16:50:54.804521 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:54.804486    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:55.071652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:55.071583    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:55.199995 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:55.199947    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-zrjc7"
Jul 20 16:50:55.264273 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:55.264097    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:56.296599 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.296563    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"]
Jul 20 16:50:56.297114 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.296812    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf" podUID="7c79f856-187b-4c0e-ae1e-c852455c9819" containerName="discovery" containerID="cri-o://d1eef268396fc779b5ba482f31e2dbfd5d6e59905102bd1f160cc41d44e9e0af" gracePeriod=30
Jul 20 16:50:56.553493 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.553432    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:56.606359 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.606329    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-kubeconfig\") pod \"7c79f856-187b-4c0e-ae1e-c852455c9819\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") "
Jul 20 16:50:56.606568 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.606380    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-csr-dns-cert\") pod \"7c79f856-187b-4c0e-ae1e-c852455c9819\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") "
Jul 20 16:50:56.606568 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.606419    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-token\") pod \"7c79f856-187b-4c0e-ae1e-c852455c9819\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") "
Jul 20 16:50:56.606568 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.606463    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c79f856-187b-4c0e-ae1e-c852455c9819-local-certs\") pod \"7c79f856-187b-4c0e-ae1e-c852455c9819\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") "
Jul 20 16:50:56.606568 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.606507    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-cacerts\") pod \"7c79f856-187b-4c0e-ae1e-c852455c9819\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") "
Jul 20 16:50:56.606568 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.606553    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-csr-ca-configmap\") pod \"7c79f856-187b-4c0e-ae1e-c852455c9819\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") "
Jul 20 16:50:56.606849 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.606575    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26m8j\" (UniqueName: \"kubernetes.io/projected/7c79f856-187b-4c0e-ae1e-c852455c9819-kube-api-access-26m8j\") pod \"7c79f856-187b-4c0e-ae1e-c852455c9819\" (UID: \"7c79f856-187b-4c0e-ae1e-c852455c9819\") "
Jul 20 16:50:56.607065 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.607021    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "7c79f856-187b-4c0e-ae1e-c852455c9819" (UID: "7c79f856-187b-4c0e-ae1e-c852455c9819"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:50:56.609783 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.609745    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-cacerts" (OuterVolumeSpecName: "cacerts") pod "7c79f856-187b-4c0e-ae1e-c852455c9819" (UID: "7c79f856-187b-4c0e-ae1e-c852455c9819"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:50:56.609886 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.609785    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "7c79f856-187b-4c0e-ae1e-c852455c9819" (UID: "7c79f856-187b-4c0e-ae1e-c852455c9819"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:50:56.610079 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.610052    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-token" (OuterVolumeSpecName: "istio-token") pod "7c79f856-187b-4c0e-ae1e-c852455c9819" (UID: "7c79f856-187b-4c0e-ae1e-c852455c9819"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:50:56.610166 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.610105    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c79f856-187b-4c0e-ae1e-c852455c9819-kube-api-access-26m8j" (OuterVolumeSpecName: "kube-api-access-26m8j") pod "7c79f856-187b-4c0e-ae1e-c852455c9819" (UID: "7c79f856-187b-4c0e-ae1e-c852455c9819"). InnerVolumeSpecName "kube-api-access-26m8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:50:56.610335 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.610309    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "7c79f856-187b-4c0e-ae1e-c852455c9819" (UID: "7c79f856-187b-4c0e-ae1e-c852455c9819"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:50:56.610798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.610779    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c79f856-187b-4c0e-ae1e-c852455c9819-local-certs" (OuterVolumeSpecName: "local-certs") pod "7c79f856-187b-4c0e-ae1e-c852455c9819" (UID: "7c79f856-187b-4c0e-ae1e-c852455c9819"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:50:56.707468 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.707434    2568 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-csr-ca-configmap\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:50:56.707468 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.707468    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26m8j\" (UniqueName: \"kubernetes.io/projected/7c79f856-187b-4c0e-ae1e-c852455c9819-kube-api-access-26m8j\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:50:56.707652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.707482    2568 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-kubeconfig\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:50:56.707652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.707497    2568 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-csr-dns-cert\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:50:56.707652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.707508    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c79f856-187b-4c0e-ae1e-c852455c9819-istio-token\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:50:56.707652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.707518    2568 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c79f856-187b-4c0e-ae1e-c852455c9819-local-certs\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:50:56.707652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:56.707526    2568 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7c79f856-187b-4c0e-ae1e-c852455c9819-cacerts\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:50:57.247715 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.247629    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp" event={"ID":"e43ba35f-7425-48b4-a81c-49dcfb086017","Type":"ContainerStarted","Data":"51926862f4a1acafc4a31f509aeb02a069d32eb54268dec3ae5484f4ca8c6cde"}
Jul 20 16:50:57.247874 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.247832    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp"
Jul 20 16:50:57.249101 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.249063    2568 generic.go:356] "Generic (PLEG): container finished" podID="7c79f856-187b-4c0e-ae1e-c852455c9819" containerID="d1eef268396fc779b5ba482f31e2dbfd5d6e59905102bd1f160cc41d44e9e0af" exitCode=0
Jul 20 16:50:57.249212 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.249115    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf" event={"ID":"7c79f856-187b-4c0e-ae1e-c852455c9819","Type":"ContainerDied","Data":"d1eef268396fc779b5ba482f31e2dbfd5d6e59905102bd1f160cc41d44e9e0af"}
Jul 20 16:50:57.249212 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.249145    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"
Jul 20 16:50:57.249212 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.249164    2568 scope.go:117] "RemoveContainer" containerID="d1eef268396fc779b5ba482f31e2dbfd5d6e59905102bd1f160cc41d44e9e0af"
Jul 20 16:50:57.249354 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.249152    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-shmkf" event={"ID":"7c79f856-187b-4c0e-ae1e-c852455c9819","Type":"ContainerDied","Data":"15a9200fce158cc3f77b976bb9c61755098e17c285121fff3e112c7fb796473a"}
Jul 20 16:50:57.286294 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.286275    2568 scope.go:117] "RemoveContainer" containerID="d1eef268396fc779b5ba482f31e2dbfd5d6e59905102bd1f160cc41d44e9e0af"
Jul 20 16:50:57.286642 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:50:57.286622    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1eef268396fc779b5ba482f31e2dbfd5d6e59905102bd1f160cc41d44e9e0af\": container with ID starting with d1eef268396fc779b5ba482f31e2dbfd5d6e59905102bd1f160cc41d44e9e0af not found: ID does not exist" containerID="d1eef268396fc779b5ba482f31e2dbfd5d6e59905102bd1f160cc41d44e9e0af"
Jul 20 16:50:57.286717 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.286653    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1eef268396fc779b5ba482f31e2dbfd5d6e59905102bd1f160cc41d44e9e0af"} err="failed to get container status \"d1eef268396fc779b5ba482f31e2dbfd5d6e59905102bd1f160cc41d44e9e0af\": rpc error: code = NotFound desc = could not find container \"d1eef268396fc779b5ba482f31e2dbfd5d6e59905102bd1f160cc41d44e9e0af\": container with ID starting with d1eef268396fc779b5ba482f31e2dbfd5d6e59905102bd1f160cc41d44e9e0af not found: ID does not exist"
Jul 20 16:50:57.364837 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.364789    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp" podStartSLOduration=1.754843345 podStartE2EDuration="4.36475285s" totalImagesPullingTime="2.609909505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:57.317391732 +0000 UTC m=+336.865247790" watchObservedRunningTime="2026-07-20 16:50:57.36475285 +0000 UTC m=+336.912608916"
Jul 20 16:50:57.366139 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.366114    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"]
Jul 20 16:50:57.368807 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.368786    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-shmkf"]
Jul 20 16:50:57.409409 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.409381    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:57.810296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:57.810257    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:50:58.254357 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:58.254323    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8" event={"ID":"5acb4774-f9cf-4e43-94b1-d990d3033b11","Type":"ContainerStarted","Data":"9ac820960bc1f8c77a9ce04d24ed01c4ce6f128935c7e43918c0e9bdc00068fa"}
Jul 20 16:50:58.254562 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:58.254463    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8"
Jul 20 16:50:58.895727 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:50:58.895696    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c79f856-187b-4c0e-ae1e-c852455c9819" path="/var/lib/kubelet/pods/7c79f856-187b-4c0e-ae1e-c852455c9819/volumes"
Jul 20 16:51:28.260902 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:28.260827    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lxdxp"
Jul 20 16:51:28.290795 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:28.290738    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8" podStartSLOduration=31.87697465 podStartE2EDuration="35.290719593s" totalImagesPullingTime="3.413744943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:58.452687599 +0000 UTC m=+338.000543667" watchObservedRunningTime="2026-07-20 16:51:28.290719593 +0000 UTC m=+367.838575724"
Jul 20 16:51:29.262154 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:29.262122    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd9c8"
Jul 20 16:51:37.637103 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.637070    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-46b2g"]
Jul 20 16:51:37.637510 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.637488    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c79f856-187b-4c0e-ae1e-c852455c9819" containerName="discovery"
Jul 20 16:51:37.637510 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.637506    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c79f856-187b-4c0e-ae1e-c852455c9819" containerName="discovery"
Jul 20 16:51:37.637611 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.637597    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c79f856-187b-4c0e-ae1e-c852455c9819" containerName="discovery"
Jul 20 16:51:37.650350 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.650323    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-46b2g"
Jul 20 16:51:37.663043 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.663022    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-5xdnd\""
Jul 20 16:51:37.664033 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.664019    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 20 16:51:37.664080 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.664049    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 20 16:51:37.673749 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.673723    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-46b2g"]
Jul 20 16:51:37.744212 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.744182    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bn7p\" (UniqueName: \"kubernetes.io/projected/6d0c6300-5ed8-44e5-ab3e-b64c8086b18c-kube-api-access-9bn7p\") pod \"kuadrant-operator-catalog-46b2g\" (UID: \"6d0c6300-5ed8-44e5-ab3e-b64c8086b18c\") " pod="kuadrant-system/kuadrant-operator-catalog-46b2g"
Jul 20 16:51:37.844826 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.844793    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bn7p\" (UniqueName: \"kubernetes.io/projected/6d0c6300-5ed8-44e5-ab3e-b64c8086b18c-kube-api-access-9bn7p\") pod \"kuadrant-operator-catalog-46b2g\" (UID: \"6d0c6300-5ed8-44e5-ab3e-b64c8086b18c\") " pod="kuadrant-system/kuadrant-operator-catalog-46b2g"
Jul 20 16:51:37.857546 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.857521    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bn7p\" (UniqueName: \"kubernetes.io/projected/6d0c6300-5ed8-44e5-ab3e-b64c8086b18c-kube-api-access-9bn7p\") pod \"kuadrant-operator-catalog-46b2g\" (UID: \"6d0c6300-5ed8-44e5-ab3e-b64c8086b18c\") " pod="kuadrant-system/kuadrant-operator-catalog-46b2g"
Jul 20 16:51:37.953345 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.953259    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-46b2g"]
Jul 20 16:51:37.953512 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:37.953498    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-46b2g"
Jul 20 16:51:38.126507 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:38.126484    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-46b2g"]
Jul 20 16:51:38.129049 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:51:38.129019    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d0c6300_5ed8_44e5_ab3e_b64c8086b18c.slice/crio-af91e23101225ebfb3344ab8fd24de63256a0935a27bd4597ef5eccaed6af765 WatchSource:0}: Error finding container af91e23101225ebfb3344ab8fd24de63256a0935a27bd4597ef5eccaed6af765: Status 404 returned error can't find the container with id af91e23101225ebfb3344ab8fd24de63256a0935a27bd4597ef5eccaed6af765
Jul 20 16:51:38.191406 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:38.191373    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-q2flc"]
Jul 20 16:51:38.195775 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:38.195753    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-q2flc"
Jul 20 16:51:38.216222 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:38.216169    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-q2flc"]
Jul 20 16:51:38.248499 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:38.248469    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fqpx\" (UniqueName: \"kubernetes.io/projected/308a46b9-9916-430e-9532-be14539dbc3d-kube-api-access-8fqpx\") pod \"kuadrant-operator-catalog-q2flc\" (UID: \"308a46b9-9916-430e-9532-be14539dbc3d\") " pod="kuadrant-system/kuadrant-operator-catalog-q2flc"
Jul 20 16:51:38.329404 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:38.329372    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:51:38.348807 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:38.348778    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fqpx\" (UniqueName: \"kubernetes.io/projected/308a46b9-9916-430e-9532-be14539dbc3d-kube-api-access-8fqpx\") pod \"kuadrant-operator-catalog-q2flc\" (UID: \"308a46b9-9916-430e-9532-be14539dbc3d\") " pod="kuadrant-system/kuadrant-operator-catalog-q2flc"
Jul 20 16:51:38.359507 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:38.359481    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fqpx\" (UniqueName: \"kubernetes.io/projected/308a46b9-9916-430e-9532-be14539dbc3d-kube-api-access-8fqpx\") pod \"kuadrant-operator-catalog-q2flc\" (UID: \"308a46b9-9916-430e-9532-be14539dbc3d\") " pod="kuadrant-system/kuadrant-operator-catalog-q2flc"
Jul 20 16:51:38.413590 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:38.413557    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:51:38.416712 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:38.416686    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-46b2g" event={"ID":"6d0c6300-5ed8-44e5-ab3e-b64c8086b18c","Type":"ContainerStarted","Data":"af91e23101225ebfb3344ab8fd24de63256a0935a27bd4597ef5eccaed6af765"}
Jul 20 16:51:38.504720 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:38.504640    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-q2flc"
Jul 20 16:51:38.638069 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:38.638042    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-q2flc"]
Jul 20 16:51:38.675002 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:51:38.674965    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308a46b9_9916_430e_9532_be14539dbc3d.slice/crio-b5f99e03a542a323815e9a023bf061b12a42dcb23e2cdf8f094385ee975a9df3 WatchSource:0}: Error finding container b5f99e03a542a323815e9a023bf061b12a42dcb23e2cdf8f094385ee975a9df3: Status 404 returned error can't find the container with id b5f99e03a542a323815e9a023bf061b12a42dcb23e2cdf8f094385ee975a9df3
Jul 20 16:51:39.423056 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:39.423017    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-q2flc" event={"ID":"308a46b9-9916-430e-9532-be14539dbc3d","Type":"ContainerStarted","Data":"b5f99e03a542a323815e9a023bf061b12a42dcb23e2cdf8f094385ee975a9df3"}
Jul 20 16:51:40.429839 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:40.429805    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-q2flc" event={"ID":"308a46b9-9916-430e-9532-be14539dbc3d","Type":"ContainerStarted","Data":"893c83ec7e72d3cee82b757e214bebda3ae40b5ede0ffdd69165d699d6a80243"}
Jul 20 16:51:40.431408 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:40.431376    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-46b2g" event={"ID":"6d0c6300-5ed8-44e5-ab3e-b64c8086b18c","Type":"ContainerStarted","Data":"93a3ec63c1770cf1399fa67c4a09b54ccad9c32e32a302c58aebf162060c333b"}
Jul 20 16:51:40.431631 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:40.431421    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-46b2g" podUID="6d0c6300-5ed8-44e5-ab3e-b64c8086b18c" containerName="registry-server" containerID="cri-o://93a3ec63c1770cf1399fa67c4a09b54ccad9c32e32a302c58aebf162060c333b" gracePeriod=2
Jul 20 16:51:40.482860 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:40.482743    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-q2flc" podStartSLOduration=0.926396823 podStartE2EDuration="2.482724677s" totalImagesPullingTime="1.556327854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:51:40.45328344 +0000 UTC m=+380.001139507" watchObservedRunningTime="2026-07-20 16:51:40.482724677 +0000 UTC m=+380.030580744"
Jul 20 16:51:40.483393 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:40.483359    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-46b2g" podStartSLOduration=1.384123857 podStartE2EDuration="3.483348692s" totalImagesPullingTime="2.099224835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:51:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:51:40.482419967 +0000 UTC m=+380.030276023" watchObservedRunningTime="2026-07-20 16:51:40.483348692 +0000 UTC m=+380.031204757"
Jul 20 16:51:40.672615 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:40.672592    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-46b2g"
Jul 20 16:51:40.768467 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:40.768381    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bn7p\" (UniqueName: \"kubernetes.io/projected/6d0c6300-5ed8-44e5-ab3e-b64c8086b18c-kube-api-access-9bn7p\") pod \"6d0c6300-5ed8-44e5-ab3e-b64c8086b18c\" (UID: \"6d0c6300-5ed8-44e5-ab3e-b64c8086b18c\") "
Jul 20 16:51:40.770587 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:40.770567    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d0c6300-5ed8-44e5-ab3e-b64c8086b18c-kube-api-access-9bn7p" (OuterVolumeSpecName: "kube-api-access-9bn7p") pod "6d0c6300-5ed8-44e5-ab3e-b64c8086b18c" (UID: "6d0c6300-5ed8-44e5-ab3e-b64c8086b18c"). InnerVolumeSpecName "kube-api-access-9bn7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:51:40.869127 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:40.869096    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bn7p\" (UniqueName: \"kubernetes.io/projected/6d0c6300-5ed8-44e5-ab3e-b64c8086b18c-kube-api-access-9bn7p\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:51:41.436857 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:41.436823    2568 generic.go:356] "Generic (PLEG): container finished" podID="6d0c6300-5ed8-44e5-ab3e-b64c8086b18c" containerID="93a3ec63c1770cf1399fa67c4a09b54ccad9c32e32a302c58aebf162060c333b" exitCode=0
Jul 20 16:51:41.437347 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:41.436889    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-46b2g"
Jul 20 16:51:41.437347 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:41.436906    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-46b2g" event={"ID":"6d0c6300-5ed8-44e5-ab3e-b64c8086b18c","Type":"ContainerDied","Data":"93a3ec63c1770cf1399fa67c4a09b54ccad9c32e32a302c58aebf162060c333b"}
Jul 20 16:51:41.437347 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:41.436951    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-46b2g" event={"ID":"6d0c6300-5ed8-44e5-ab3e-b64c8086b18c","Type":"ContainerDied","Data":"af91e23101225ebfb3344ab8fd24de63256a0935a27bd4597ef5eccaed6af765"}
Jul 20 16:51:41.437347 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:41.436971    2568 scope.go:117] "RemoveContainer" containerID="93a3ec63c1770cf1399fa67c4a09b54ccad9c32e32a302c58aebf162060c333b"
Jul 20 16:51:41.445394 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:41.445377    2568 scope.go:117] "RemoveContainer" containerID="93a3ec63c1770cf1399fa67c4a09b54ccad9c32e32a302c58aebf162060c333b"
Jul 20 16:51:41.445672 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:51:41.445652    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93a3ec63c1770cf1399fa67c4a09b54ccad9c32e32a302c58aebf162060c333b\": container with ID starting with 93a3ec63c1770cf1399fa67c4a09b54ccad9c32e32a302c58aebf162060c333b not found: ID does not exist" containerID="93a3ec63c1770cf1399fa67c4a09b54ccad9c32e32a302c58aebf162060c333b"
Jul 20 16:51:41.445717 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:41.445682    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93a3ec63c1770cf1399fa67c4a09b54ccad9c32e32a302c58aebf162060c333b"} err="failed to get container status \"93a3ec63c1770cf1399fa67c4a09b54ccad9c32e32a302c58aebf162060c333b\": rpc error: code = NotFound desc = could not find container \"93a3ec63c1770cf1399fa67c4a09b54ccad9c32e32a302c58aebf162060c333b\": container with ID starting with 93a3ec63c1770cf1399fa67c4a09b54ccad9c32e32a302c58aebf162060c333b not found: ID does not exist"
Jul 20 16:51:41.454678 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:41.454658    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-46b2g"]
Jul 20 16:51:41.459819 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:41.459798    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-46b2g"]
Jul 20 16:51:42.895086 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:42.895045    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d0c6300-5ed8-44e5-ab3e-b64c8086b18c" path="/var/lib/kubelet/pods/6d0c6300-5ed8-44e5-ab3e-b64c8086b18c/volumes"
Jul 20 16:51:48.505481 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:48.505448    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-q2flc"
Jul 20 16:51:48.505876 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:48.505504    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-q2flc"
Jul 20 16:51:48.527771 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:48.527742    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-q2flc"
Jul 20 16:51:49.493573 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:51:49.493541    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-q2flc"
Jul 20 16:52:06.603087 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:06.603036    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:08.430907 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:08.430879    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:08.530098 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:08.530070    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:08.630865 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:08.630837    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:10.107059 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:10.107020    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:10.210340 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:10.210305    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:11.628939 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.628905    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"]
Jul 20 16:52:11.629356 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.629220    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0c6300-5ed8-44e5-ab3e-b64c8086b18c" containerName="registry-server"
Jul 20 16:52:11.629356 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.629232    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0c6300-5ed8-44e5-ab3e-b64c8086b18c" containerName="registry-server"
Jul 20 16:52:11.629356 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.629306    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d0c6300-5ed8-44e5-ab3e-b64c8086b18c" containerName="registry-server"
Jul 20 16:52:11.633865 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.633837    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:11.637524 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.637507    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:11.642437 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.642399    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-l28jj\""
Jul 20 16:52:11.656401 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.656376    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"]
Jul 20 16:52:11.730284 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.730211    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzn9f\" (UniqueName: \"kubernetes.io/projected/b2bbb981-cfe6-4a64-9448-8c9794d799a8-kube-api-access-vzn9f\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" (UID: \"b2bbb981-cfe6-4a64-9448-8c9794d799a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:11.730460 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.730371    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2bbb981-cfe6-4a64-9448-8c9794d799a8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" (UID: \"b2bbb981-cfe6-4a64-9448-8c9794d799a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:11.831230 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.831191    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2bbb981-cfe6-4a64-9448-8c9794d799a8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" (UID: \"b2bbb981-cfe6-4a64-9448-8c9794d799a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:11.831433 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.831272    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzn9f\" (UniqueName: \"kubernetes.io/projected/b2bbb981-cfe6-4a64-9448-8c9794d799a8-kube-api-access-vzn9f\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" (UID: \"b2bbb981-cfe6-4a64-9448-8c9794d799a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:11.831670 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.831647    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2bbb981-cfe6-4a64-9448-8c9794d799a8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" (UID: \"b2bbb981-cfe6-4a64-9448-8c9794d799a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:11.847558 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.847527    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzn9f\" (UniqueName: \"kubernetes.io/projected/b2bbb981-cfe6-4a64-9448-8c9794d799a8-kube-api-access-vzn9f\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" (UID: \"b2bbb981-cfe6-4a64-9448-8c9794d799a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:11.947228 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:11.947140    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:12.039831 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:12.039805    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:12.105833 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:12.105799    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"]
Jul 20 16:52:12.106014 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:52:12.105994    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2bbb981_cfe6_4a64_9448_8c9794d799a8.slice/crio-f52cbc6faecc3e28fb5d09f1758c33a52e6af37fb10c72529cb4555c2f27b6b4 WatchSource:0}: Error finding container f52cbc6faecc3e28fb5d09f1758c33a52e6af37fb10c72529cb4555c2f27b6b4: Status 404 returned error can't find the container with id f52cbc6faecc3e28fb5d09f1758c33a52e6af37fb10c72529cb4555c2f27b6b4
Jul 20 16:52:12.563924 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:12.563883    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6" event={"ID":"b2bbb981-cfe6-4a64-9448-8c9794d799a8","Type":"ContainerStarted","Data":"f52cbc6faecc3e28fb5d09f1758c33a52e6af37fb10c72529cb4555c2f27b6b4"}
Jul 20 16:52:13.058693 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:13.058649    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:13.164718 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:13.164673    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:14.083817 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:14.083784    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dksk2"]
Jul 20 16:52:14.087196 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:14.087174    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dksk2"
Jul 20 16:52:14.091608 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:14.091589    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-slgc4\""
Jul 20 16:52:14.113786 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:14.113757    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dksk2"]
Jul 20 16:52:14.251358 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:14.251324    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-658nn\" (UniqueName: \"kubernetes.io/projected/24cfeb48-8cde-4a53-9d68-0eb5c6973ab7-kube-api-access-658nn\") pod \"authorino-operator-75cc8d5cb7-dksk2\" (UID: \"24cfeb48-8cde-4a53-9d68-0eb5c6973ab7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dksk2"
Jul 20 16:52:14.351872 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:14.351777    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-658nn\" (UniqueName: \"kubernetes.io/projected/24cfeb48-8cde-4a53-9d68-0eb5c6973ab7-kube-api-access-658nn\") pod \"authorino-operator-75cc8d5cb7-dksk2\" (UID: \"24cfeb48-8cde-4a53-9d68-0eb5c6973ab7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dksk2"
Jul 20 16:52:14.361936 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:14.361895    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-658nn\" (UniqueName: \"kubernetes.io/projected/24cfeb48-8cde-4a53-9d68-0eb5c6973ab7-kube-api-access-658nn\") pod \"authorino-operator-75cc8d5cb7-dksk2\" (UID: \"24cfeb48-8cde-4a53-9d68-0eb5c6973ab7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dksk2"
Jul 20 16:52:14.397402 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:14.397362    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dksk2"
Jul 20 16:52:14.538710 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:14.538679    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dksk2"]
Jul 20 16:52:14.541629 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:52:14.541601    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24cfeb48_8cde_4a53_9d68_0eb5c6973ab7.slice/crio-26c3ec81e6f97901907841af9d896c4622b8498a9611d7b6201f00a8c21d8b64 WatchSource:0}: Error finding container 26c3ec81e6f97901907841af9d896c4622b8498a9611d7b6201f00a8c21d8b64: Status 404 returned error can't find the container with id 26c3ec81e6f97901907841af9d896c4622b8498a9611d7b6201f00a8c21d8b64
Jul 20 16:52:14.574139 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:14.574107    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dksk2" event={"ID":"24cfeb48-8cde-4a53-9d68-0eb5c6973ab7","Type":"ContainerStarted","Data":"26c3ec81e6f97901907841af9d896c4622b8498a9611d7b6201f00a8c21d8b64"}
Jul 20 16:52:14.659554 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:14.659520    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:14.800328 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:14.800299    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:16.263567 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:16.263533    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"]
Jul 20 16:52:16.272603 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:16.272581    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"
Jul 20 16:52:16.275995 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:16.275971    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5dbq2\""
Jul 20 16:52:16.276897 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:16.276870    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 20 16:52:16.281669 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:16.281058    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"]
Jul 20 16:52:16.369663 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:16.369619    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nrfv\" (UniqueName: \"kubernetes.io/projected/9351b069-61d0-490c-a90b-a8b68fa167dd-kube-api-access-9nrfv\") pod \"dns-operator-controller-manager-864dd78c4c-hfghk\" (UID: \"9351b069-61d0-490c-a90b-a8b68fa167dd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"
Jul 20 16:52:16.471028 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:16.470998    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nrfv\" (UniqueName: \"kubernetes.io/projected/9351b069-61d0-490c-a90b-a8b68fa167dd-kube-api-access-9nrfv\") pod \"dns-operator-controller-manager-864dd78c4c-hfghk\" (UID: \"9351b069-61d0-490c-a90b-a8b68fa167dd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"
Jul 20 16:52:16.485280 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:16.485229    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nrfv\" (UniqueName: \"kubernetes.io/projected/9351b069-61d0-490c-a90b-a8b68fa167dd-kube-api-access-9nrfv\") pod \"dns-operator-controller-manager-864dd78c4c-hfghk\" (UID: \"9351b069-61d0-490c-a90b-a8b68fa167dd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"
Jul 20 16:52:16.585646 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:16.585574    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"
Jul 20 16:52:16.814356 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:16.814321    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:16.979209 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:16.979184    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"]
Jul 20 16:52:16.980153 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:52:16.980122    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9351b069_61d0_490c_a90b_a8b68fa167dd.slice/crio-66f3319c610f0aab94726c8937690624d387e2d5980413179ad8ad88aa945139 WatchSource:0}: Error finding container 66f3319c610f0aab94726c8937690624d387e2d5980413179ad8ad88aa945139: Status 404 returned error can't find the container with id 66f3319c610f0aab94726c8937690624d387e2d5980413179ad8ad88aa945139
Jul 20 16:52:17.590517 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:17.590483    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dksk2" event={"ID":"24cfeb48-8cde-4a53-9d68-0eb5c6973ab7","Type":"ContainerStarted","Data":"dc458339e0860b700a21c20b7faa163f61db05bb25d974d4fe27f2b74250e540"}
Jul 20 16:52:17.590980 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:17.590575    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dksk2"
Jul 20 16:52:17.591705 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:17.591675    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk" event={"ID":"9351b069-61d0-490c-a90b-a8b68fa167dd","Type":"ContainerStarted","Data":"66f3319c610f0aab94726c8937690624d387e2d5980413179ad8ad88aa945139"}
Jul 20 16:52:17.617871 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:17.614716    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dksk2" podStartSLOduration=1.271044591 podStartE2EDuration="3.614694807s" totalImagesPullingTime="2.343650216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:52:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:52:17.613350638 +0000 UTC m=+417.161206706" watchObservedRunningTime="2026-07-20 16:52:17.614694807 +0000 UTC m=+417.162550875"
Jul 20 16:52:18.244571 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:18.244536    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"]
Jul 20 16:52:18.248217 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:18.248190    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:18.250950 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:18.250925    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-qgqkm\""
Jul 20 16:52:18.264142 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:18.264120    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"]
Jul 20 16:52:18.388657 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:18.388615    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9rb8\" (UniqueName: \"kubernetes.io/projected/1b520131-804b-4713-ae9e-3df1c8adf055-kube-api-access-b9rb8\") pod \"limitador-operator-controller-manager-859d7b78bd-p57dw\" (UID: \"1b520131-804b-4713-ae9e-3df1c8adf055\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:18.489692 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:18.489649    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9rb8\" (UniqueName: \"kubernetes.io/projected/1b520131-804b-4713-ae9e-3df1c8adf055-kube-api-access-b9rb8\") pod \"limitador-operator-controller-manager-859d7b78bd-p57dw\" (UID: \"1b520131-804b-4713-ae9e-3df1c8adf055\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:18.502042 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:18.501967    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9rb8\" (UniqueName: \"kubernetes.io/projected/1b520131-804b-4713-ae9e-3df1c8adf055-kube-api-access-b9rb8\") pod \"limitador-operator-controller-manager-859d7b78bd-p57dw\" (UID: \"1b520131-804b-4713-ae9e-3df1c8adf055\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:18.560431 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:18.560398    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:18.685526 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:18.685445    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:19.038774 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:19.038723    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:19.140157 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:19.140126    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:19.246056 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:19.245997    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:19.375026 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:19.374984    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:19.951342 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:19.951311    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"]
Jul 20 16:52:19.953994 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:52:19.953951    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b520131_804b_4713_ae9e_3df1c8adf055.slice/crio-9651662e8330eaa0cb46fe386f3ffe1911e573784008a2d0f9d2941aa3ce1d31 WatchSource:0}: Error finding container 9651662e8330eaa0cb46fe386f3ffe1911e573784008a2d0f9d2941aa3ce1d31: Status 404 returned error can't find the container with id 9651662e8330eaa0cb46fe386f3ffe1911e573784008a2d0f9d2941aa3ce1d31
Jul 20 16:52:20.415114 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:20.415078    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:20.515212 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:20.515175    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:20.611482 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:20.611443    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw" event={"ID":"1b520131-804b-4713-ae9e-3df1c8adf055","Type":"ContainerStarted","Data":"9651662e8330eaa0cb46fe386f3ffe1911e573784008a2d0f9d2941aa3ce1d31"}
Jul 20 16:52:20.614003 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:20.613595    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6" event={"ID":"b2bbb981-cfe6-4a64-9448-8c9794d799a8","Type":"ContainerStarted","Data":"5f3601e774dafe199cc5888799a9e5389f204cffac1db96127a40271b7ff96f2"}
Jul 20 16:52:20.614003 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:20.613702    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:20.615741 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:20.615713    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk" event={"ID":"9351b069-61d0-490c-a90b-a8b68fa167dd","Type":"ContainerStarted","Data":"1ae74c0d2ef22947b8f09e3549064c86b5b9bc673b395e6326c9d1ed4482349f"}
Jul 20 16:52:20.615866 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:20.615853    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"
Jul 20 16:52:20.637839 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:20.637733    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6" podStartSLOduration=1.936832952 podStartE2EDuration="9.637709003s" totalImagesPullingTime="7.700876051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:52:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:52:20.63712598 +0000 UTC m=+420.184982050" watchObservedRunningTime="2026-07-20 16:52:20.637709003 +0000 UTC m=+420.185565070"
Jul 20 16:52:20.657546 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:20.657475    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk" podStartSLOduration=1.777542125 podStartE2EDuration="4.65745285s" totalImagesPullingTime="2.879910725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:52:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:52:20.65711755 +0000 UTC m=+420.204973616" watchObservedRunningTime="2026-07-20 16:52:20.65745285 +0000 UTC m=+420.205308917"
Jul 20 16:52:22.537636 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:22.537599    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:22.627256 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:22.627213    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw" event={"ID":"1b520131-804b-4713-ae9e-3df1c8adf055","Type":"ContainerStarted","Data":"454255524bcd1bc6ca61258d83da29ed52ebd24fed5690023676a0005f56f198"}
Jul 20 16:52:22.627440 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:22.627336    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:22.696360 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:22.696308    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw" podStartSLOduration=2.945070853 podStartE2EDuration="4.696293034s" totalImagesPullingTime="1.751222181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:52:22.695176837 +0000 UTC m=+422.243032903" watchObservedRunningTime="2026-07-20 16:52:22.696293034 +0000 UTC m=+422.244149100"
Jul 20 16:52:22.786195 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:22.786157    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:23.094941 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.094908    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"]
Jul 20 16:52:23.098340 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.098322    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"
Jul 20 16:52:23.104347 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.104315    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 20 16:52:23.111350 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.111319    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 20 16:52:23.111350 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.111337    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h9v4b\""
Jul 20 16:52:23.119060 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.119038    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"]
Jul 20 16:52:23.125991 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.125968    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2929cae6-b7cf-4740-b628-635c1caf86d0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4g64s\" (UID: \"2929cae6-b7cf-4740-b628-635c1caf86d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"
Jul 20 16:52:23.126101 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.126026    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2929cae6-b7cf-4740-b628-635c1caf86d0-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4g64s\" (UID: \"2929cae6-b7cf-4740-b628-635c1caf86d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"
Jul 20 16:52:23.126101 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.126092    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6h5m\" (UniqueName: \"kubernetes.io/projected/2929cae6-b7cf-4740-b628-635c1caf86d0-kube-api-access-m6h5m\") pod \"kuadrant-console-plugin-66b5774985-4g64s\" (UID: \"2929cae6-b7cf-4740-b628-635c1caf86d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"
Jul 20 16:52:23.188128 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.188096    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:23.226975 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.226936    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2929cae6-b7cf-4740-b628-635c1caf86d0-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4g64s\" (UID: \"2929cae6-b7cf-4740-b628-635c1caf86d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"
Jul 20 16:52:23.227199 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.226986    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6h5m\" (UniqueName: \"kubernetes.io/projected/2929cae6-b7cf-4740-b628-635c1caf86d0-kube-api-access-m6h5m\") pod \"kuadrant-console-plugin-66b5774985-4g64s\" (UID: \"2929cae6-b7cf-4740-b628-635c1caf86d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"
Jul 20 16:52:23.227199 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.227029    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2929cae6-b7cf-4740-b628-635c1caf86d0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4g64s\" (UID: \"2929cae6-b7cf-4740-b628-635c1caf86d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"
Jul 20 16:52:23.227749 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.227722    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2929cae6-b7cf-4740-b628-635c1caf86d0-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4g64s\" (UID: \"2929cae6-b7cf-4740-b628-635c1caf86d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"
Jul 20 16:52:23.229655 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.229636    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2929cae6-b7cf-4740-b628-635c1caf86d0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4g64s\" (UID: \"2929cae6-b7cf-4740-b628-635c1caf86d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"
Jul 20 16:52:23.241753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.241726    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6h5m\" (UniqueName: \"kubernetes.io/projected/2929cae6-b7cf-4740-b628-635c1caf86d0-kube-api-access-m6h5m\") pod \"kuadrant-console-plugin-66b5774985-4g64s\" (UID: \"2929cae6-b7cf-4740-b628-635c1caf86d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"
Jul 20 16:52:23.349493 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.349381    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:23.407360 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.407324    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"
Jul 20 16:52:23.574862 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.574831    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s"]
Jul 20 16:52:23.578342 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:52:23.578317    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2929cae6_b7cf_4740_b628_635c1caf86d0.slice/crio-761181af7371aad5edbc102a8b907b1d65c79389b2629e997a3fd76009e1bd18 WatchSource:0}: Error finding container 761181af7371aad5edbc102a8b907b1d65c79389b2629e997a3fd76009e1bd18: Status 404 returned error can't find the container with id 761181af7371aad5edbc102a8b907b1d65c79389b2629e997a3fd76009e1bd18
Jul 20 16:52:23.632713 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:23.632685    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s" event={"ID":"2929cae6-b7cf-4740-b628-635c1caf86d0","Type":"ContainerStarted","Data":"761181af7371aad5edbc102a8b907b1d65c79389b2629e997a3fd76009e1bd18"}
Jul 20 16:52:28.600732 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:28.600696    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dksk2"
Jul 20 16:52:29.555529 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:29.555491    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:29.776200 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:29.776165    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:29.945851 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:29.945818    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:31.623483 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:31.623449    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:31.624001 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:31.623825    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"
Jul 20 16:52:31.711924 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:31.711829    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:31.826986 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:31.826948    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:33.635834 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.635799    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:33.640318 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.640180    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"]
Jul 20 16:52:33.640493 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.640450    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" containerName="manager" containerID="cri-o://5f3601e774dafe199cc5888799a9e5389f204cffac1db96127a40271b7ff96f2" gracePeriod=2
Jul 20 16:52:33.642155 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.642129    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"]
Jul 20 16:52:33.655982 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.655952    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"]
Jul 20 16:52:33.672181 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.671952    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"]
Jul 20 16:52:33.673603 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.673569    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"]
Jul 20 16:52:33.674015 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.673994    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b520131-804b-4713-ae9e-3df1c8adf055" containerName="manager"
Jul 20 16:52:33.674152 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.674139    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b520131-804b-4713-ae9e-3df1c8adf055" containerName="manager"
Jul 20 16:52:33.674292 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.674274    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" containerName="manager"
Jul 20 16:52:33.674292 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.674290    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" containerName="manager"
Jul 20 16:52:33.674511 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.674382    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" containerName="manager"
Jul 20 16:52:33.674511 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.674391    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b520131-804b-4713-ae9e-3df1c8adf055" containerName="manager"
Jul 20 16:52:33.678480 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.678457    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"
Jul 20 16:52:33.685590 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.685222    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw" podUID="1b520131-804b-4713-ae9e-3df1c8adf055" containerName="manager" containerID="cri-o://454255524bcd1bc6ca61258d83da29ed52ebd24fed5690023676a0005f56f198" gracePeriod=2
Jul 20 16:52:33.687994 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.687967    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"]
Jul 20 16:52:33.693670 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.693644    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"
Jul 20 16:52:33.693670 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.693659    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"]
Jul 20 16:52:33.708524 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.708450    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"]
Jul 20 16:52:33.726106 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.726070    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmw4m\" (UniqueName: \"kubernetes.io/projected/954a9eb2-1593-4471-abf5-f30e49776c4c-kube-api-access-xmw4m\") pod \"limitador-operator-controller-manager-859d7b78bd-f2n7m\" (UID: \"954a9eb2-1593-4471-abf5-f30e49776c4c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"
Jul 20 16:52:33.726291 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.726133    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-656l5\" (UniqueName: \"kubernetes.io/projected/0b92e4df-9c4d-4eef-8fef-81fbbea944af-kube-api-access-656l5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xkjpb\" (UID: \"0b92e4df-9c4d-4eef-8fef-81fbbea944af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"
Jul 20 16:52:33.726291 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.726168    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b92e4df-9c4d-4eef-8fef-81fbbea944af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xkjpb\" (UID: \"0b92e4df-9c4d-4eef-8fef-81fbbea944af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"
Jul 20 16:52:33.726463 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:33.726432    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:33.751885 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:33.751847    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p57dw\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="1b520131-804b-4713-ae9e-3df1c8adf055" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:33.753805 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:33.753777    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:33.827036 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.827000    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b92e4df-9c4d-4eef-8fef-81fbbea944af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xkjpb\" (UID: \"0b92e4df-9c4d-4eef-8fef-81fbbea944af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"
Jul 20 16:52:33.827201 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.827100    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmw4m\" (UniqueName: \"kubernetes.io/projected/954a9eb2-1593-4471-abf5-f30e49776c4c-kube-api-access-xmw4m\") pod \"limitador-operator-controller-manager-859d7b78bd-f2n7m\" (UID: \"954a9eb2-1593-4471-abf5-f30e49776c4c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"
Jul 20 16:52:33.827201 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.827143    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-656l5\" (UniqueName: \"kubernetes.io/projected/0b92e4df-9c4d-4eef-8fef-81fbbea944af-kube-api-access-656l5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xkjpb\" (UID: \"0b92e4df-9c4d-4eef-8fef-81fbbea944af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"
Jul 20 16:52:33.827475 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.827445    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b92e4df-9c4d-4eef-8fef-81fbbea944af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xkjpb\" (UID: \"0b92e4df-9c4d-4eef-8fef-81fbbea944af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"
Jul 20 16:52:33.840661 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.840631    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-656l5\" (UniqueName: \"kubernetes.io/projected/0b92e4df-9c4d-4eef-8fef-81fbbea944af-kube-api-access-656l5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xkjpb\" (UID: \"0b92e4df-9c4d-4eef-8fef-81fbbea944af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"
Jul 20 16:52:33.840791 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.840727    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmw4m\" (UniqueName: \"kubernetes.io/projected/954a9eb2-1593-4471-abf5-f30e49776c4c-kube-api-access-xmw4m\") pod \"limitador-operator-controller-manager-859d7b78bd-f2n7m\" (UID: \"954a9eb2-1593-4471-abf5-f30e49776c4c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"
Jul 20 16:52:33.946780 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.946699    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"]
Jul 20 16:52:33.955339 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.955310    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"
Jul 20 16:52:33.965919 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:33.965891    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"]
Jul 20 16:52:33.996575 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:33.996537    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p57dw\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="1b520131-804b-4713-ae9e-3df1c8adf055" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:33.998663 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:33.998629    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:34.029129 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:34.029091    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa6fda9f-5098-4028-b6ec-61a9795d3667-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ct47n\" (UID: \"aa6fda9f-5098-4028-b6ec-61a9795d3667\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"
Jul 20 16:52:34.029318 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:34.029203    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppzfb\" (UniqueName: \"kubernetes.io/projected/aa6fda9f-5098-4028-b6ec-61a9795d3667-kube-api-access-ppzfb\") pod \"kuadrant-operator-controller-manager-86878c769f-ct47n\" (UID: \"aa6fda9f-5098-4028-b6ec-61a9795d3667\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"
Jul 20 16:52:34.078720 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:34.078679    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"
Jul 20 16:52:34.084925 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:34.084897    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"
Jul 20 16:52:34.130041 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:34.130000    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa6fda9f-5098-4028-b6ec-61a9795d3667-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ct47n\" (UID: \"aa6fda9f-5098-4028-b6ec-61a9795d3667\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"
Jul 20 16:52:34.130235 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:34.130067    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppzfb\" (UniqueName: \"kubernetes.io/projected/aa6fda9f-5098-4028-b6ec-61a9795d3667-kube-api-access-ppzfb\") pod \"kuadrant-operator-controller-manager-86878c769f-ct47n\" (UID: \"aa6fda9f-5098-4028-b6ec-61a9795d3667\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"
Jul 20 16:52:34.130479 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:34.130452    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa6fda9f-5098-4028-b6ec-61a9795d3667-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ct47n\" (UID: \"aa6fda9f-5098-4028-b6ec-61a9795d3667\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"
Jul 20 16:52:34.142437 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:34.142413    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppzfb\" (UniqueName: \"kubernetes.io/projected/aa6fda9f-5098-4028-b6ec-61a9795d3667-kube-api-access-ppzfb\") pod \"kuadrant-operator-controller-manager-86878c769f-ct47n\" (UID: \"aa6fda9f-5098-4028-b6ec-61a9795d3667\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"
Jul 20 16:52:34.269200 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:34.269132    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"
Jul 20 16:52:36.452396 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:36.452366    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:36.549691 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:36.549653    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:40.254734 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:40.254693    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:40.393993 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:40.393934    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:46.622716 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.622693    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"]
Jul 20 16:52:46.625099 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:52:46.625070    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod954a9eb2_1593_4471_abf5_f30e49776c4c.slice/crio-5e46f2e970c6753e5e98bcb4cbc23f04ab020b76506159f3baf648d5a0f8efe5 WatchSource:0}: Error finding container 5e46f2e970c6753e5e98bcb4cbc23f04ab020b76506159f3baf648d5a0f8efe5: Status 404 returned error can't find the container with id 5e46f2e970c6753e5e98bcb4cbc23f04ab020b76506159f3baf648d5a0f8efe5
Jul 20 16:52:46.626781 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.626757    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:46.627207 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.627177    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"]
Jul 20 16:52:46.627821 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:52:46.627789    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa6fda9f_5098_4028_b6ec_61a9795d3667.slice/crio-e6b225bafea9799736f942fcfa6c38c74e436307766ae5566aa77e11b7d10abb WatchSource:0}: Error finding container e6b225bafea9799736f942fcfa6c38c74e436307766ae5566aa77e11b7d10abb: Status 404 returned error can't find the container with id e6b225bafea9799736f942fcfa6c38c74e436307766ae5566aa77e11b7d10abb
Jul 20 16:52:46.629362 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:46.629322    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p57dw\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="1b520131-804b-4713-ae9e-3df1c8adf055" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:46.631886 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.631865    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"]
Jul 20 16:52:46.632920 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:52:46.632899    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b92e4df_9c4d_4eef_8fef_81fbbea944af.slice/crio-1eb43505c5c9d7c6b84efa0a00df42c6b975cecf5881ee049bd8d10b73e83814 WatchSource:0}: Error finding container 1eb43505c5c9d7c6b84efa0a00df42c6b975cecf5881ee049bd8d10b73e83814: Status 404 returned error can't find the container with id 1eb43505c5c9d7c6b84efa0a00df42c6b975cecf5881ee049bd8d10b73e83814
Jul 20 16:52:46.657257 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.657218    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:46.659850 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:46.659820    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:46.661992 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:46.661969    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p57dw\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="1b520131-804b-4713-ae9e-3df1c8adf055" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:46.751073 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.750767    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9rb8\" (UniqueName: \"kubernetes.io/projected/1b520131-804b-4713-ae9e-3df1c8adf055-kube-api-access-b9rb8\") pod \"1b520131-804b-4713-ae9e-3df1c8adf055\" (UID: \"1b520131-804b-4713-ae9e-3df1c8adf055\") "
Jul 20 16:52:46.751073 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.750859    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2bbb981-cfe6-4a64-9448-8c9794d799a8-extensions-socket-volume\") pod \"b2bbb981-cfe6-4a64-9448-8c9794d799a8\" (UID: \"b2bbb981-cfe6-4a64-9448-8c9794d799a8\") "
Jul 20 16:52:46.751073 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.750940    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzn9f\" (UniqueName: \"kubernetes.io/projected/b2bbb981-cfe6-4a64-9448-8c9794d799a8-kube-api-access-vzn9f\") pod \"b2bbb981-cfe6-4a64-9448-8c9794d799a8\" (UID: \"b2bbb981-cfe6-4a64-9448-8c9794d799a8\") "
Jul 20 16:52:46.751734 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.751672    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2bbb981-cfe6-4a64-9448-8c9794d799a8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b2bbb981-cfe6-4a64-9448-8c9794d799a8" (UID: "b2bbb981-cfe6-4a64-9448-8c9794d799a8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:52:46.751834 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.751780    2568 generic.go:356] "Generic (PLEG): container finished" podID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" containerID="5f3601e774dafe199cc5888799a9e5389f204cffac1db96127a40271b7ff96f2" exitCode=0
Jul 20 16:52:46.751896 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.751847    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:46.751977 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.751854    2568 scope.go:117] "RemoveContainer" containerID="5f3601e774dafe199cc5888799a9e5389f204cffac1db96127a40271b7ff96f2"
Jul 20 16:52:46.753435 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.753409    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b520131-804b-4713-ae9e-3df1c8adf055-kube-api-access-b9rb8" (OuterVolumeSpecName: "kube-api-access-b9rb8") pod "1b520131-804b-4713-ae9e-3df1c8adf055" (UID: "1b520131-804b-4713-ae9e-3df1c8adf055"). InnerVolumeSpecName "kube-api-access-b9rb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:52:46.753910 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.753853    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb" event={"ID":"0b92e4df-9c4d-4eef-8fef-81fbbea944af","Type":"ContainerStarted","Data":"1eb43505c5c9d7c6b84efa0a00df42c6b975cecf5881ee049bd8d10b73e83814"}
Jul 20 16:52:46.754486 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:46.754460    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:46.754861 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.754828    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2bbb981-cfe6-4a64-9448-8c9794d799a8-kube-api-access-vzn9f" (OuterVolumeSpecName: "kube-api-access-vzn9f") pod "b2bbb981-cfe6-4a64-9448-8c9794d799a8" (UID: "b2bbb981-cfe6-4a64-9448-8c9794d799a8"). InnerVolumeSpecName "kube-api-access-vzn9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:52:46.755865 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.755843    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m" event={"ID":"954a9eb2-1593-4471-abf5-f30e49776c4c","Type":"ContainerStarted","Data":"26d22076be752d24418cd2c6b17bbfd65f1a20890aa28326284f133a44c44ade"}
Jul 20 16:52:46.755965 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.755896    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m" event={"ID":"954a9eb2-1593-4471-abf5-f30e49776c4c","Type":"ContainerStarted","Data":"5e46f2e970c6753e5e98bcb4cbc23f04ab020b76506159f3baf648d5a0f8efe5"}
Jul 20 16:52:46.757114 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:46.757089    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p57dw\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="1b520131-804b-4713-ae9e-3df1c8adf055" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:46.758017 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.757989    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n" event={"ID":"aa6fda9f-5098-4028-b6ec-61a9795d3667","Type":"ContainerStarted","Data":"e6b225bafea9799736f942fcfa6c38c74e436307766ae5566aa77e11b7d10abb"}
Jul 20 16:52:46.759382 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.759363    2568 generic.go:356] "Generic (PLEG): container finished" podID="1b520131-804b-4713-ae9e-3df1c8adf055" containerID="454255524bcd1bc6ca61258d83da29ed52ebd24fed5690023676a0005f56f198" exitCode=0
Jul 20 16:52:46.759464 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.759416    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:46.761885 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:46.761859    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:46.762316 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.762291    2568 scope.go:117] "RemoveContainer" containerID="5f3601e774dafe199cc5888799a9e5389f204cffac1db96127a40271b7ff96f2"
Jul 20 16:52:46.762632 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:46.762603    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f3601e774dafe199cc5888799a9e5389f204cffac1db96127a40271b7ff96f2\": container with ID starting with 5f3601e774dafe199cc5888799a9e5389f204cffac1db96127a40271b7ff96f2 not found: ID does not exist" containerID="5f3601e774dafe199cc5888799a9e5389f204cffac1db96127a40271b7ff96f2"
Jul 20 16:52:46.762738 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.762643    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f3601e774dafe199cc5888799a9e5389f204cffac1db96127a40271b7ff96f2"} err="failed to get container status \"5f3601e774dafe199cc5888799a9e5389f204cffac1db96127a40271b7ff96f2\": rpc error: code = NotFound desc = could not find container \"5f3601e774dafe199cc5888799a9e5389f204cffac1db96127a40271b7ff96f2\": container with ID starting with 5f3601e774dafe199cc5888799a9e5389f204cffac1db96127a40271b7ff96f2 not found: ID does not exist"
Jul 20 16:52:46.762738 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.762666    2568 scope.go:117] "RemoveContainer" containerID="454255524bcd1bc6ca61258d83da29ed52ebd24fed5690023676a0005f56f198"
Jul 20 16:52:46.764087 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:46.764059    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p57dw\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="1b520131-804b-4713-ae9e-3df1c8adf055" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:46.772730 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.772714    2568 scope.go:117] "RemoveContainer" containerID="454255524bcd1bc6ca61258d83da29ed52ebd24fed5690023676a0005f56f198"
Jul 20 16:52:46.772979 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:46.772962    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"454255524bcd1bc6ca61258d83da29ed52ebd24fed5690023676a0005f56f198\": container with ID starting with 454255524bcd1bc6ca61258d83da29ed52ebd24fed5690023676a0005f56f198 not found: ID does not exist" containerID="454255524bcd1bc6ca61258d83da29ed52ebd24fed5690023676a0005f56f198"
Jul 20 16:52:46.773021 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.772986    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454255524bcd1bc6ca61258d83da29ed52ebd24fed5690023676a0005f56f198"} err="failed to get container status \"454255524bcd1bc6ca61258d83da29ed52ebd24fed5690023676a0005f56f198\": rpc error: code = NotFound desc = could not find container \"454255524bcd1bc6ca61258d83da29ed52ebd24fed5690023676a0005f56f198\": container with ID starting with 454255524bcd1bc6ca61258d83da29ed52ebd24fed5690023676a0005f56f198 not found: ID does not exist"
Jul 20 16:52:46.781605 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:46.781578    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p57dw\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="1b520131-804b-4713-ae9e-3df1c8adf055" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p57dw"
Jul 20 16:52:46.783790 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:46.783769    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:46.852704 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.852634    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2bbb981-cfe6-4a64-9448-8c9794d799a8-extensions-socket-volume\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:52:46.852704 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.852668    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzn9f\" (UniqueName: \"kubernetes.io/projected/b2bbb981-cfe6-4a64-9448-8c9794d799a8-kube-api-access-vzn9f\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:52:46.852704 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.852683    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9rb8\" (UniqueName: \"kubernetes.io/projected/1b520131-804b-4713-ae9e-3df1c8adf055-kube-api-access-b9rb8\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:52:46.895614 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.895584    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b520131-804b-4713-ae9e-3df1c8adf055" path="/var/lib/kubelet/pods/1b520131-804b-4713-ae9e-3df1c8adf055/volumes"
Jul 20 16:52:46.895937 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:46.895924    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" path="/var/lib/kubelet/pods/b2bbb981-cfe6-4a64-9448-8c9794d799a8/volumes"
Jul 20 16:52:47.057753 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:47.057721    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:47.765324 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:47.765283    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n" event={"ID":"aa6fda9f-5098-4028-b6ec-61a9795d3667","Type":"ContainerStarted","Data":"06990e9f72151fb3b49d2f77ce1439e0c025997a644be3779389ed6ad7b49f61"}
Jul 20 16:52:47.765896 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:47.765399    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"
Jul 20 16:52:47.767523 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:47.767500    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s" event={"ID":"2929cae6-b7cf-4740-b628-635c1caf86d0","Type":"ContainerStarted","Data":"9a79be3daf6d15833211dc094364f4ccc2d5d89c6a061b2ce4c2f49b78da27ad"}
Jul 20 16:52:47.767782 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:47.767751    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:47.769808 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:47.769788    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb" event={"ID":"0b92e4df-9c4d-4eef-8fef-81fbbea944af","Type":"ContainerStarted","Data":"b3b14133891b8cfdab70903e3865de5c4d4722318e3e39a8a1ffc2567f6b50ce"}
Jul 20 16:52:47.769881 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:47.769839    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"
Jul 20 16:52:47.791028 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:47.790989    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n" podStartSLOduration=14.790975366 podStartE2EDuration="14.790975366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:52:47.788778203 +0000 UTC m=+447.336634269" watchObservedRunningTime="2026-07-20 16:52:47.790975366 +0000 UTC m=+447.338831431"
Jul 20 16:52:47.791591 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:47.791554    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wljm6\" is forbidden: User \"system:node:ip-10-0-128-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-8.ec2.internal' and this object" podUID="b2bbb981-cfe6-4a64-9448-8c9794d799a8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wljm6"
Jul 20 16:52:47.815317 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:47.815273    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb" podStartSLOduration=14.815234817 podStartE2EDuration="14.815234817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:52:47.814153927 +0000 UTC m=+447.362009994" watchObservedRunningTime="2026-07-20 16:52:47.815234817 +0000 UTC m=+447.363090884"
Jul 20 16:52:47.835533 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:47.835487    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m" podStartSLOduration=14.835472808 podStartE2EDuration="14.835472808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:52:47.834181448 +0000 UTC m=+447.382037527" watchObservedRunningTime="2026-07-20 16:52:47.835472808 +0000 UTC m=+447.383328873"
Jul 20 16:52:47.852815 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:47.852769    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4g64s" podStartSLOduration=1.78010471 podStartE2EDuration="24.852752732s" totalImagesPullingTime="23.072648022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:52:47.850585028 +0000 UTC m=+447.398441085" watchObservedRunningTime="2026-07-20 16:52:47.852752732 +0000 UTC m=+447.400608800"
Jul 20 16:52:49.007802 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:49.007767    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:49.109038 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:49.109000    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:51.463107 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:51.463074    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:51.561162 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:51.561128    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:52.928064 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:52.928031    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:56.229757 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:56.229726    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:56.330070 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:56.330037    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:57.770314 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:57.770279    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"
Jul 20 16:52:57.772276 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:57.772255    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"
Jul 20 16:52:58.780564 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:58.780532    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"
Jul 20 16:52:58.781015 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:58.780596    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"
Jul 20 16:52:58.911143 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:58.911113    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"]
Jul 20 16:52:58.911336 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:58.911316    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb" podUID="0b92e4df-9c4d-4eef-8fef-81fbbea944af" containerName="manager" containerID="cri-o://b3b14133891b8cfdab70903e3865de5c4d4722318e3e39a8a1ffc2567f6b50ce" gracePeriod=10
Jul 20 16:52:59.169179 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.169157    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"
Jul 20 16:52:59.257859 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.257835    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b92e4df-9c4d-4eef-8fef-81fbbea944af-extensions-socket-volume\") pod \"0b92e4df-9c4d-4eef-8fef-81fbbea944af\" (UID: \"0b92e4df-9c4d-4eef-8fef-81fbbea944af\") "
Jul 20 16:52:59.258054 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.257867    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-656l5\" (UniqueName: \"kubernetes.io/projected/0b92e4df-9c4d-4eef-8fef-81fbbea944af-kube-api-access-656l5\") pod \"0b92e4df-9c4d-4eef-8fef-81fbbea944af\" (UID: \"0b92e4df-9c4d-4eef-8fef-81fbbea944af\") "
Jul 20 16:52:59.258331 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.258295    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b92e4df-9c4d-4eef-8fef-81fbbea944af-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0b92e4df-9c4d-4eef-8fef-81fbbea944af" (UID: "0b92e4df-9c4d-4eef-8fef-81fbbea944af"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:52:59.260161 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.260137    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b92e4df-9c4d-4eef-8fef-81fbbea944af-kube-api-access-656l5" (OuterVolumeSpecName: "kube-api-access-656l5") pod "0b92e4df-9c4d-4eef-8fef-81fbbea944af" (UID: "0b92e4df-9c4d-4eef-8fef-81fbbea944af"). InnerVolumeSpecName "kube-api-access-656l5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:52:59.358892 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.358816    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b92e4df-9c4d-4eef-8fef-81fbbea944af-extensions-socket-volume\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:52:59.358892 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.358843    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-656l5\" (UniqueName: \"kubernetes.io/projected/0b92e4df-9c4d-4eef-8fef-81fbbea944af-kube-api-access-656l5\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:52:59.828069 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.828035    2568 generic.go:356] "Generic (PLEG): container finished" podID="0b92e4df-9c4d-4eef-8fef-81fbbea944af" containerID="b3b14133891b8cfdab70903e3865de5c4d4722318e3e39a8a1ffc2567f6b50ce" exitCode=0
Jul 20 16:52:59.828570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.828087    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"
Jul 20 16:52:59.828570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.828128    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb" event={"ID":"0b92e4df-9c4d-4eef-8fef-81fbbea944af","Type":"ContainerDied","Data":"b3b14133891b8cfdab70903e3865de5c4d4722318e3e39a8a1ffc2567f6b50ce"}
Jul 20 16:52:59.828570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.828210    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb" event={"ID":"0b92e4df-9c4d-4eef-8fef-81fbbea944af","Type":"ContainerDied","Data":"1eb43505c5c9d7c6b84efa0a00df42c6b975cecf5881ee049bd8d10b73e83814"}
Jul 20 16:52:59.828570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.828250    2568 scope.go:117] "RemoveContainer" containerID="b3b14133891b8cfdab70903e3865de5c4d4722318e3e39a8a1ffc2567f6b50ce"
Jul 20 16:52:59.837178 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.837161    2568 scope.go:117] "RemoveContainer" containerID="b3b14133891b8cfdab70903e3865de5c4d4722318e3e39a8a1ffc2567f6b50ce"
Jul 20 16:52:59.837455 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:52:59.837433    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3b14133891b8cfdab70903e3865de5c4d4722318e3e39a8a1ffc2567f6b50ce\": container with ID starting with b3b14133891b8cfdab70903e3865de5c4d4722318e3e39a8a1ffc2567f6b50ce not found: ID does not exist" containerID="b3b14133891b8cfdab70903e3865de5c4d4722318e3e39a8a1ffc2567f6b50ce"
Jul 20 16:52:59.837529 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.837463    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3b14133891b8cfdab70903e3865de5c4d4722318e3e39a8a1ffc2567f6b50ce"} err="failed to get container status \"b3b14133891b8cfdab70903e3865de5c4d4722318e3e39a8a1ffc2567f6b50ce\": rpc error: code = NotFound desc = could not find container \"b3b14133891b8cfdab70903e3865de5c4d4722318e3e39a8a1ffc2567f6b50ce\": container with ID starting with b3b14133891b8cfdab70903e3865de5c4d4722318e3e39a8a1ffc2567f6b50ce not found: ID does not exist"
Jul 20 16:52:59.849605 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.849583    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"]
Jul 20 16:52:59.860051 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:52:59.860028    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xkjpb"]
Jul 20 16:53:00.610803 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:00.610764    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:00.713715 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:00.713687    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:00.896029 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:00.896001    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b92e4df-9c4d-4eef-8fef-81fbbea944af" path="/var/lib/kubelet/pods/0b92e4df-9c4d-4eef-8fef-81fbbea944af/volumes"
Jul 20 16:53:05.482300 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:05.482267    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:10.621785 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:10.621751    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:10.688776 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:10.688743    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:11.422865 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:11.422832    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:11.503054 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:11.503027    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:15.253798 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:15.253762    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:15.343560 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:15.343518    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:15.445740 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:15.445705    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:15.544614 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:15.544522    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:17.967184 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:17.967151    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:18.096642 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:18.096593    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:18.227899 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:18.227802    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:18.857164 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:18.857133    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-54d5x"]
Jul 20 16:53:18.857506 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:18.857493    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b92e4df-9c4d-4eef-8fef-81fbbea944af" containerName="manager"
Jul 20 16:53:18.857562 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:18.857507    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b92e4df-9c4d-4eef-8fef-81fbbea944af" containerName="manager"
Jul 20 16:53:18.857596 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:18.857578    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b92e4df-9c4d-4eef-8fef-81fbbea944af" containerName="manager"
Jul 20 16:53:18.890172 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:18.890149    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-54d5x"]
Jul 20 16:53:18.890358 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:18.890297    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-54d5x"
Jul 20 16:53:18.893037 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:18.892982    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:18.893700 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:18.893675    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gcwqs\""
Jul 20 16:53:18.908829 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:18.908807    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkcx5\" (UniqueName: \"kubernetes.io/projected/eb0b787d-73f6-45fd-878c-c7d41e0a3a67-kube-api-access-jkcx5\") pod \"authorino-58cd955675-54d5x\" (UID: \"eb0b787d-73f6-45fd-878c-c7d41e0a3a67\") " pod="kuadrant-system/authorino-58cd955675-54d5x"
Jul 20 16:53:19.009641 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:19.009614    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkcx5\" (UniqueName: \"kubernetes.io/projected/eb0b787d-73f6-45fd-878c-c7d41e0a3a67-kube-api-access-jkcx5\") pod \"authorino-58cd955675-54d5x\" (UID: \"eb0b787d-73f6-45fd-878c-c7d41e0a3a67\") " pod="kuadrant-system/authorino-58cd955675-54d5x"
Jul 20 16:53:19.022636 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:19.022604    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkcx5\" (UniqueName: \"kubernetes.io/projected/eb0b787d-73f6-45fd-878c-c7d41e0a3a67-kube-api-access-jkcx5\") pod \"authorino-58cd955675-54d5x\" (UID: \"eb0b787d-73f6-45fd-878c-c7d41e0a3a67\") " pod="kuadrant-system/authorino-58cd955675-54d5x"
Jul 20 16:53:19.176544 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:19.176502    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:19.199559 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:19.199532    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-54d5x"
Jul 20 16:53:19.336030 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:19.336005    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-54d5x"]
Jul 20 16:53:19.337988 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:53:19.337961    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb0b787d_73f6_45fd_878c_c7d41e0a3a67.slice/crio-799df8fb1dc6ba17f0e27dc797db9592ca930bf6a0d5933bdab8284d64c9aa45 WatchSource:0}: Error finding container 799df8fb1dc6ba17f0e27dc797db9592ca930bf6a0d5933bdab8284d64c9aa45: Status 404 returned error can't find the container with id 799df8fb1dc6ba17f0e27dc797db9592ca930bf6a0d5933bdab8284d64c9aa45
Jul 20 16:53:19.633154 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:19.633118    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:19.748201 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:19.748161    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:19.921834 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:19.921748    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-54d5x" event={"ID":"eb0b787d-73f6-45fd-878c-c7d41e0a3a67","Type":"ContainerStarted","Data":"799df8fb1dc6ba17f0e27dc797db9592ca930bf6a0d5933bdab8284d64c9aa45"}
Jul 20 16:53:22.115427 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:22.115393    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:22.380611 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:22.380572    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:22.941492 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:22.941456    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-54d5x" event={"ID":"eb0b787d-73f6-45fd-878c-c7d41e0a3a67","Type":"ContainerStarted","Data":"3e0ed5934d42bf277cf1e44420a99102ab426f2c3db59da25d2b7e6d4acbf6a1"}
Jul 20 16:53:22.959021 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:22.958928    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-54d5x" podStartSLOduration=1.592275264 podStartE2EDuration="4.958913725s" totalImagesPullingTime="3.366638461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:53:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:53:22.958080235 +0000 UTC m=+482.505936302" watchObservedRunningTime="2026-07-20 16:53:22.958913725 +0000 UTC m=+482.506769791"
Jul 20 16:53:24.251762 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:24.251733    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-54d5x"]
Jul 20 16:53:24.950341 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:24.950299    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-54d5x" podUID="eb0b787d-73f6-45fd-878c-c7d41e0a3a67" containerName="authorino" containerID="cri-o://3e0ed5934d42bf277cf1e44420a99102ab426f2c3db59da25d2b7e6d4acbf6a1" gracePeriod=30
Jul 20 16:53:25.190570 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:25.190550    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-54d5x"
Jul 20 16:53:25.261208 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:25.261133    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkcx5\" (UniqueName: \"kubernetes.io/projected/eb0b787d-73f6-45fd-878c-c7d41e0a3a67-kube-api-access-jkcx5\") pod \"eb0b787d-73f6-45fd-878c-c7d41e0a3a67\" (UID: \"eb0b787d-73f6-45fd-878c-c7d41e0a3a67\") "
Jul 20 16:53:25.263269 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:25.263234    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb0b787d-73f6-45fd-878c-c7d41e0a3a67-kube-api-access-jkcx5" (OuterVolumeSpecName: "kube-api-access-jkcx5") pod "eb0b787d-73f6-45fd-878c-c7d41e0a3a67" (UID: "eb0b787d-73f6-45fd-878c-c7d41e0a3a67"). InnerVolumeSpecName "kube-api-access-jkcx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:53:25.362437 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:25.362410    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkcx5\" (UniqueName: \"kubernetes.io/projected/eb0b787d-73f6-45fd-878c-c7d41e0a3a67-kube-api-access-jkcx5\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:53:25.955889 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:25.955860    2568 generic.go:356] "Generic (PLEG): container finished" podID="eb0b787d-73f6-45fd-878c-c7d41e0a3a67" containerID="3e0ed5934d42bf277cf1e44420a99102ab426f2c3db59da25d2b7e6d4acbf6a1" exitCode=0
Jul 20 16:53:25.956074 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:25.955896    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-54d5x" event={"ID":"eb0b787d-73f6-45fd-878c-c7d41e0a3a67","Type":"ContainerDied","Data":"3e0ed5934d42bf277cf1e44420a99102ab426f2c3db59da25d2b7e6d4acbf6a1"}
Jul 20 16:53:25.956074 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:25.955911    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-54d5x"
Jul 20 16:53:25.956074 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:25.955931    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-54d5x" event={"ID":"eb0b787d-73f6-45fd-878c-c7d41e0a3a67","Type":"ContainerDied","Data":"799df8fb1dc6ba17f0e27dc797db9592ca930bf6a0d5933bdab8284d64c9aa45"}
Jul 20 16:53:25.956074 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:25.955953    2568 scope.go:117] "RemoveContainer" containerID="3e0ed5934d42bf277cf1e44420a99102ab426f2c3db59da25d2b7e6d4acbf6a1"
Jul 20 16:53:25.964512 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:25.964493    2568 scope.go:117] "RemoveContainer" containerID="3e0ed5934d42bf277cf1e44420a99102ab426f2c3db59da25d2b7e6d4acbf6a1"
Jul 20 16:53:25.964745 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:53:25.964728    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e0ed5934d42bf277cf1e44420a99102ab426f2c3db59da25d2b7e6d4acbf6a1\": container with ID starting with 3e0ed5934d42bf277cf1e44420a99102ab426f2c3db59da25d2b7e6d4acbf6a1 not found: ID does not exist" containerID="3e0ed5934d42bf277cf1e44420a99102ab426f2c3db59da25d2b7e6d4acbf6a1"
Jul 20 16:53:25.964796 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:25.964751    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e0ed5934d42bf277cf1e44420a99102ab426f2c3db59da25d2b7e6d4acbf6a1"} err="failed to get container status \"3e0ed5934d42bf277cf1e44420a99102ab426f2c3db59da25d2b7e6d4acbf6a1\": rpc error: code = NotFound desc = could not find container \"3e0ed5934d42bf277cf1e44420a99102ab426f2c3db59da25d2b7e6d4acbf6a1\": container with ID starting with 3e0ed5934d42bf277cf1e44420a99102ab426f2c3db59da25d2b7e6d4acbf6a1 not found: ID does not exist"
Jul 20 16:53:25.979405 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:25.979381    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-54d5x"]
Jul 20 16:53:25.983017 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:25.982997    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-54d5x"]
Jul 20 16:53:26.895853 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:26.895819    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb0b787d-73f6-45fd-878c-c7d41e0a3a67" path="/var/lib/kubelet/pods/eb0b787d-73f6-45fd-878c-c7d41e0a3a67/volumes"
Jul 20 16:53:33.870175 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:33.870144    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:33.963398 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:33.963360    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:34.356476 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:34.356381    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:46.655295 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:46.655256    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:46.980473 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:46.980396    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:53.206068 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.206029    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:53.233082 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.233052    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b578ccfcd-kp69x"]
Jul 20 16:53:53.233472 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.233455    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb0b787d-73f6-45fd-878c-c7d41e0a3a67" containerName="authorino"
Jul 20 16:53:53.233472 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.233473    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb0b787d-73f6-45fd-878c-c7d41e0a3a67" containerName="authorino"
Jul 20 16:53:53.233611 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.233563    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb0b787d-73f6-45fd-878c-c7d41e0a3a67" containerName="authorino"
Jul 20 16:53:53.236646 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.236629    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b578ccfcd-kp69x"
Jul 20 16:53:53.241139 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.241118    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 16:53:53.241284 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.241161    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 16:53:53.241284 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.241164    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7lpl2\""
Jul 20 16:53:53.252627 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.252603    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b578ccfcd-kp69x"]
Jul 20 16:53:53.337860 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.337828    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:53.385755 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.385728    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce633f47-5f17-412a-a850-4cfe00ca3b4f-webhook-cert\") pod \"maas-controller-6b578ccfcd-kp69x\" (UID: \"ce633f47-5f17-412a-a850-4cfe00ca3b4f\") " pod="opendatahub/maas-controller-6b578ccfcd-kp69x"
Jul 20 16:53:53.385856 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.385767    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvb9z\" (UniqueName: \"kubernetes.io/projected/ce633f47-5f17-412a-a850-4cfe00ca3b4f-kube-api-access-dvb9z\") pod \"maas-controller-6b578ccfcd-kp69x\" (UID: \"ce633f47-5f17-412a-a850-4cfe00ca3b4f\") " pod="opendatahub/maas-controller-6b578ccfcd-kp69x"
Jul 20 16:53:53.430446 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.430418    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:53.435130 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.435109    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-qw2qm"]
Jul 20 16:53:53.438268 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.438254    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77b8bc97bf-qw2qm"
Jul 20 16:53:53.460888 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.460839    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-qw2qm"]
Jul 20 16:53:53.486611 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.486586    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvb9z\" (UniqueName: \"kubernetes.io/projected/ce633f47-5f17-412a-a850-4cfe00ca3b4f-kube-api-access-dvb9z\") pod \"maas-controller-6b578ccfcd-kp69x\" (UID: \"ce633f47-5f17-412a-a850-4cfe00ca3b4f\") " pod="opendatahub/maas-controller-6b578ccfcd-kp69x"
Jul 20 16:53:53.486723 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.486664    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce633f47-5f17-412a-a850-4cfe00ca3b4f-webhook-cert\") pod \"maas-controller-6b578ccfcd-kp69x\" (UID: \"ce633f47-5f17-412a-a850-4cfe00ca3b4f\") " pod="opendatahub/maas-controller-6b578ccfcd-kp69x"
Jul 20 16:53:53.486777 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:53:53.486763    2568 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 20 16:53:53.486833 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:53:53.486824    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce633f47-5f17-412a-a850-4cfe00ca3b4f-webhook-cert podName:ce633f47-5f17-412a-a850-4cfe00ca3b4f nodeName:}" failed. No retries permitted until 2026-07-20 16:53:53.986806221 +0000 UTC m=+513.534662277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/ce633f47-5f17-412a-a850-4cfe00ca3b4f-webhook-cert") pod "maas-controller-6b578ccfcd-kp69x" (UID: "ce633f47-5f17-412a-a850-4cfe00ca3b4f") : secret "maas-controller-webhook-cert" not found
Jul 20 16:53:53.497166 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.497142    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvb9z\" (UniqueName: \"kubernetes.io/projected/ce633f47-5f17-412a-a850-4cfe00ca3b4f-kube-api-access-dvb9z\") pod \"maas-controller-6b578ccfcd-kp69x\" (UID: \"ce633f47-5f17-412a-a850-4cfe00ca3b4f\") " pod="opendatahub/maas-controller-6b578ccfcd-kp69x"
Jul 20 16:53:53.587053 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.587021    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvwbs\" (UniqueName: \"kubernetes.io/projected/4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0-kube-api-access-vvwbs\") pod \"maas-controller-77b8bc97bf-qw2qm\" (UID: \"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0\") " pod="opendatahub/maas-controller-77b8bc97bf-qw2qm"
Jul 20 16:53:53.587184 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.587083    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0-webhook-cert\") pod \"maas-controller-77b8bc97bf-qw2qm\" (UID: \"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0\") " pod="opendatahub/maas-controller-77b8bc97bf-qw2qm"
Jul 20 16:53:53.687639 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.687610    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0-webhook-cert\") pod \"maas-controller-77b8bc97bf-qw2qm\" (UID: \"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0\") " pod="opendatahub/maas-controller-77b8bc97bf-qw2qm"
Jul 20 16:53:53.687822 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.687723    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvwbs\" (UniqueName: \"kubernetes.io/projected/4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0-kube-api-access-vvwbs\") pod \"maas-controller-77b8bc97bf-qw2qm\" (UID: \"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0\") " pod="opendatahub/maas-controller-77b8bc97bf-qw2qm"
Jul 20 16:53:53.689303 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.689275    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:53.690523 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.690504    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0-webhook-cert\") pod \"maas-controller-77b8bc97bf-qw2qm\" (UID: \"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0\") " pod="opendatahub/maas-controller-77b8bc97bf-qw2qm"
Jul 20 16:53:53.696881 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.696859    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvwbs\" (UniqueName: \"kubernetes.io/projected/4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0-kube-api-access-vvwbs\") pod \"maas-controller-77b8bc97bf-qw2qm\" (UID: \"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0\") " pod="opendatahub/maas-controller-77b8bc97bf-qw2qm"
Jul 20 16:53:53.748489 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.748392    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77b8bc97bf-qw2qm"
Jul 20 16:53:53.906392 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.906369    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-qw2qm"]
Jul 20 16:53:53.908214 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:53:53.908190    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f0bb7bd_17e6_4c04_ad35_dca6063ce8c0.slice/crio-1cb04456ac9dd0921132ed643816ca4c985112c20cad624555a75b73960681d1 WatchSource:0}: Error finding container 1cb04456ac9dd0921132ed643816ca4c985112c20cad624555a75b73960681d1: Status 404 returned error can't find the container with id 1cb04456ac9dd0921132ed643816ca4c985112c20cad624555a75b73960681d1
Jul 20 16:53:53.991120 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.991088    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce633f47-5f17-412a-a850-4cfe00ca3b4f-webhook-cert\") pod \"maas-controller-6b578ccfcd-kp69x\" (UID: \"ce633f47-5f17-412a-a850-4cfe00ca3b4f\") " pod="opendatahub/maas-controller-6b578ccfcd-kp69x"
Jul 20 16:53:53.993502 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:53.993479    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce633f47-5f17-412a-a850-4cfe00ca3b4f-webhook-cert\") pod \"maas-controller-6b578ccfcd-kp69x\" (UID: \"ce633f47-5f17-412a-a850-4cfe00ca3b4f\") " pod="opendatahub/maas-controller-6b578ccfcd-kp69x"
Jul 20 16:53:54.080868 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:54.080794    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77b8bc97bf-qw2qm" event={"ID":"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0","Type":"ContainerStarted","Data":"1cb04456ac9dd0921132ed643816ca4c985112c20cad624555a75b73960681d1"}
Jul 20 16:53:54.146687 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:54.146653    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b578ccfcd-kp69x"
Jul 20 16:53:54.316167 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:54.316141    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b578ccfcd-kp69x"]
Jul 20 16:53:54.318378 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:53:54.318354    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce633f47_5f17_412a_a850_4cfe00ca3b4f.slice/crio-b21cb359a62a411ff092836c9eb146f71fc274164e4e6cb74972894020ebb102 WatchSource:0}: Error finding container b21cb359a62a411ff092836c9eb146f71fc274164e4e6cb74972894020ebb102: Status 404 returned error can't find the container with id b21cb359a62a411ff092836c9eb146f71fc274164e4e6cb74972894020ebb102
Jul 20 16:53:55.090212 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:55.090173    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b578ccfcd-kp69x" event={"ID":"ce633f47-5f17-412a-a850-4cfe00ca3b4f","Type":"ContainerStarted","Data":"b21cb359a62a411ff092836c9eb146f71fc274164e4e6cb74972894020ebb102"}
Jul 20 16:53:57.022962 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:57.022925    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:57.101445 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:57.101359    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77b8bc97bf-qw2qm" event={"ID":"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0","Type":"ContainerStarted","Data":"b459c6b4f604325553ded319d1fad846600a2b397561679ed92c8dd47b77e16e"}
Jul 20 16:53:57.101617 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:57.101484    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77b8bc97bf-qw2qm"
Jul 20 16:53:57.102820 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:57.102785    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b578ccfcd-kp69x" event={"ID":"ce633f47-5f17-412a-a850-4cfe00ca3b4f","Type":"ContainerStarted","Data":"cd0e963b4c3f96635687f1484bd9a8c52ecfa426f4e4bc6c40cd2ae7c4f6b388"}
Jul 20 16:53:57.102919 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:57.102900    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b578ccfcd-kp69x"
Jul 20 16:53:57.122652 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:57.122603    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77b8bc97bf-qw2qm" podStartSLOduration=1.192099826 podStartE2EDuration="4.122588384s" totalImagesPullingTime="2.930488558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:53:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:53:57.121800075 +0000 UTC m=+516.669656139" watchObservedRunningTime="2026-07-20 16:53:57.122588384 +0000 UTC m=+516.670444450"
Jul 20 16:53:57.125699 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:53:57.125666    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:08.109104 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:08.109074    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b578ccfcd-kp69x"
Jul 20 16:54:08.109557 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:08.109142    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77b8bc97bf-qw2qm"
Jul 20 16:54:08.128286 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:08.128212    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b578ccfcd-kp69x" podStartSLOduration=12.617601581 podStartE2EDuration="15.12819643s" totalImagesPullingTime="2.510594849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:53:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:53:57.145980474 +0000 UTC m=+516.693836550" watchObservedRunningTime="2026-07-20 16:54:08.12819643 +0000 UTC m=+527.676052501"
Jul 20 16:54:08.171563 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:08.171532    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b578ccfcd-kp69x"]
Jul 20 16:54:08.171765 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:08.171740    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b578ccfcd-kp69x" podUID="ce633f47-5f17-412a-a850-4cfe00ca3b4f" containerName="manager" containerID="cri-o://cd0e963b4c3f96635687f1484bd9a8c52ecfa426f4e4bc6c40cd2ae7c4f6b388" gracePeriod=30
Jul 20 16:54:08.415383 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:08.415362    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b578ccfcd-kp69x"
Jul 20 16:54:08.611144 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:08.611114    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvb9z\" (UniqueName: \"kubernetes.io/projected/ce633f47-5f17-412a-a850-4cfe00ca3b4f-kube-api-access-dvb9z\") pod \"ce633f47-5f17-412a-a850-4cfe00ca3b4f\" (UID: \"ce633f47-5f17-412a-a850-4cfe00ca3b4f\") "
Jul 20 16:54:08.611335 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:08.611160    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce633f47-5f17-412a-a850-4cfe00ca3b4f-webhook-cert\") pod \"ce633f47-5f17-412a-a850-4cfe00ca3b4f\" (UID: \"ce633f47-5f17-412a-a850-4cfe00ca3b4f\") "
Jul 20 16:54:08.613454 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:08.613428    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce633f47-5f17-412a-a850-4cfe00ca3b4f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ce633f47-5f17-412a-a850-4cfe00ca3b4f" (UID: "ce633f47-5f17-412a-a850-4cfe00ca3b4f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:54:08.613454 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:08.613440    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce633f47-5f17-412a-a850-4cfe00ca3b4f-kube-api-access-dvb9z" (OuterVolumeSpecName: "kube-api-access-dvb9z") pod "ce633f47-5f17-412a-a850-4cfe00ca3b4f" (UID: "ce633f47-5f17-412a-a850-4cfe00ca3b4f"). InnerVolumeSpecName "kube-api-access-dvb9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:54:08.712080 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:08.712012    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvb9z\" (UniqueName: \"kubernetes.io/projected/ce633f47-5f17-412a-a850-4cfe00ca3b4f-kube-api-access-dvb9z\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:54:08.712080 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:08.712037    2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce633f47-5f17-412a-a850-4cfe00ca3b4f-webhook-cert\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:54:09.157174 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:09.157141    2568 generic.go:356] "Generic (PLEG): container finished" podID="ce633f47-5f17-412a-a850-4cfe00ca3b4f" containerID="cd0e963b4c3f96635687f1484bd9a8c52ecfa426f4e4bc6c40cd2ae7c4f6b388" exitCode=0
Jul 20 16:54:09.157616 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:09.157182    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b578ccfcd-kp69x"
Jul 20 16:54:09.157616 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:09.157222    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b578ccfcd-kp69x" event={"ID":"ce633f47-5f17-412a-a850-4cfe00ca3b4f","Type":"ContainerDied","Data":"cd0e963b4c3f96635687f1484bd9a8c52ecfa426f4e4bc6c40cd2ae7c4f6b388"}
Jul 20 16:54:09.157616 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:09.157308    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b578ccfcd-kp69x" event={"ID":"ce633f47-5f17-412a-a850-4cfe00ca3b4f","Type":"ContainerDied","Data":"b21cb359a62a411ff092836c9eb146f71fc274164e4e6cb74972894020ebb102"}
Jul 20 16:54:09.157616 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:09.157343    2568 scope.go:117] "RemoveContainer" containerID="cd0e963b4c3f96635687f1484bd9a8c52ecfa426f4e4bc6c40cd2ae7c4f6b388"
Jul 20 16:54:09.165446 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:09.165431    2568 scope.go:117] "RemoveContainer" containerID="cd0e963b4c3f96635687f1484bd9a8c52ecfa426f4e4bc6c40cd2ae7c4f6b388"
Jul 20 16:54:09.165693 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:54:09.165673    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd0e963b4c3f96635687f1484bd9a8c52ecfa426f4e4bc6c40cd2ae7c4f6b388\": container with ID starting with cd0e963b4c3f96635687f1484bd9a8c52ecfa426f4e4bc6c40cd2ae7c4f6b388 not found: ID does not exist" containerID="cd0e963b4c3f96635687f1484bd9a8c52ecfa426f4e4bc6c40cd2ae7c4f6b388"
Jul 20 16:54:09.165750 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:09.165700    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd0e963b4c3f96635687f1484bd9a8c52ecfa426f4e4bc6c40cd2ae7c4f6b388"} err="failed to get container status \"cd0e963b4c3f96635687f1484bd9a8c52ecfa426f4e4bc6c40cd2ae7c4f6b388\": rpc error: code = NotFound desc = could not find container \"cd0e963b4c3f96635687f1484bd9a8c52ecfa426f4e4bc6c40cd2ae7c4f6b388\": container with ID starting with cd0e963b4c3f96635687f1484bd9a8c52ecfa426f4e4bc6c40cd2ae7c4f6b388 not found: ID does not exist"
Jul 20 16:54:09.173867 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:09.173844    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b578ccfcd-kp69x"]
Jul 20 16:54:09.179598 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:09.179578    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b578ccfcd-kp69x"]
Jul 20 16:54:10.899481 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:10.899450    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce633f47-5f17-412a-a850-4cfe00ca3b4f" path="/var/lib/kubelet/pods/ce633f47-5f17-412a-a850-4cfe00ca3b4f/volumes"
Jul 20 16:54:26.583786 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:26.583750    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:26.724795 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:26.724763    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:27.100706 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.100668    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:27.206887 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.206857    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"]
Jul 20 16:54:27.207208 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.207196    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce633f47-5f17-412a-a850-4cfe00ca3b4f" containerName="manager"
Jul 20 16:54:27.207281 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.207210    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce633f47-5f17-412a-a850-4cfe00ca3b4f" containerName="manager"
Jul 20 16:54:27.207326 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.207308    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce633f47-5f17-412a-a850-4cfe00ca3b4f" containerName="manager"
Jul 20 16:54:27.213962 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.213940    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"
Jul 20 16:54:27.217899 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.217879    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gw9dt\""
Jul 20 16:54:27.217899 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.217892    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 20 16:54:27.218096 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.217928    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 16:54:27.218096 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.217884    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 20 16:54:27.223170 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.223136    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"]
Jul 20 16:54:27.243225 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.243195    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qg6k\" (UniqueName: \"kubernetes.io/projected/2cde84b1-5fcf-47d5-8d7e-19a872432489-kube-api-access-2qg6k\") pod \"maas-api-5df756f6bf-tfpb6\" (UID: \"2cde84b1-5fcf-47d5-8d7e-19a872432489\") " pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"
Jul 20 16:54:27.243424 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.243264    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cde84b1-5fcf-47d5-8d7e-19a872432489-maas-api-tls\") pod \"maas-api-5df756f6bf-tfpb6\" (UID: \"2cde84b1-5fcf-47d5-8d7e-19a872432489\") " pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"
Jul 20 16:54:27.243424 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.243302    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cde84b1-5fcf-47d5-8d7e-19a872432489-openshift-service-ca\") pod \"maas-api-5df756f6bf-tfpb6\" (UID: \"2cde84b1-5fcf-47d5-8d7e-19a872432489\") " pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"
Jul 20 16:54:27.298418 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.298378    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:27.344545 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.344508    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qg6k\" (UniqueName: \"kubernetes.io/projected/2cde84b1-5fcf-47d5-8d7e-19a872432489-kube-api-access-2qg6k\") pod \"maas-api-5df756f6bf-tfpb6\" (UID: \"2cde84b1-5fcf-47d5-8d7e-19a872432489\") " pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"
Jul 20 16:54:27.344545 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.344549    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cde84b1-5fcf-47d5-8d7e-19a872432489-maas-api-tls\") pod \"maas-api-5df756f6bf-tfpb6\" (UID: \"2cde84b1-5fcf-47d5-8d7e-19a872432489\") " pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"
Jul 20 16:54:27.344761 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.344611    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cde84b1-5fcf-47d5-8d7e-19a872432489-openshift-service-ca\") pod \"maas-api-5df756f6bf-tfpb6\" (UID: \"2cde84b1-5fcf-47d5-8d7e-19a872432489\") " pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"
Jul 20 16:54:27.345256 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.345217    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cde84b1-5fcf-47d5-8d7e-19a872432489-openshift-service-ca\") pod \"maas-api-5df756f6bf-tfpb6\" (UID: \"2cde84b1-5fcf-47d5-8d7e-19a872432489\") " pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"
Jul 20 16:54:27.347135 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.347116    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cde84b1-5fcf-47d5-8d7e-19a872432489-maas-api-tls\") pod \"maas-api-5df756f6bf-tfpb6\" (UID: \"2cde84b1-5fcf-47d5-8d7e-19a872432489\") " pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"
Jul 20 16:54:27.355673 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.355614    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qg6k\" (UniqueName: \"kubernetes.io/projected/2cde84b1-5fcf-47d5-8d7e-19a872432489-kube-api-access-2qg6k\") pod \"maas-api-5df756f6bf-tfpb6\" (UID: \"2cde84b1-5fcf-47d5-8d7e-19a872432489\") " pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"
Jul 20 16:54:27.529464 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.529432    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"
Jul 20 16:54:27.671904 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.671871    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"]
Jul 20 16:54:27.674162 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:54:27.674133    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cde84b1_5fcf_47d5_8d7e_19a872432489.slice/crio-e9d0166a65f645535ef769220f7acb6e76515361dd3e4c43defd72eecee8d160 WatchSource:0}: Error finding container e9d0166a65f645535ef769220f7acb6e76515361dd3e4c43defd72eecee8d160: Status 404 returned error can't find the container with id e9d0166a65f645535ef769220f7acb6e76515361dd3e4c43defd72eecee8d160
Jul 20 16:54:27.702529 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.702498    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:27.803469 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.803441    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:27.813726 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.813695    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-9kjtn"]
Jul 20 16:54:27.818961 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.818943    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn"
Jul 20 16:54:27.821485 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.821470    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 20 16:54:27.821576 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.821534    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-mf6zj\""
Jul 20 16:54:27.828801 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.828779    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-9kjtn"]
Jul 20 16:54:27.830655 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.830636    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-ffpqs"]
Jul 20 16:54:27.834491 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.834472    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-ffpqs"
Jul 20 16:54:27.842515 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.842493    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-ffpqs"]
Jul 20 16:54:27.849162 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.849137    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwhzm\" (UniqueName: \"kubernetes.io/projected/dc696b70-0020-4372-baf8-0457fefc2593-kube-api-access-bwhzm\") pod \"payload-pre-processing-7677b774bb-9kjtn\" (UID: \"dc696b70-0020-4372-baf8-0457fefc2593\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn"
Jul 20 16:54:27.849280 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.849263    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e638f7d-358e-444f-87d4-dc7c40e95588-plugins-config-volume\") pod \"payload-processing-5bf47cd988-ffpqs\" (UID: \"3e638f7d-358e-444f-87d4-dc7c40e95588\") " pod="openshift-ingress/payload-processing-5bf47cd988-ffpqs"
Jul 20 16:54:27.849400 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.849302    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc696b70-0020-4372-baf8-0457fefc2593-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-9kjtn\" (UID: \"dc696b70-0020-4372-baf8-0457fefc2593\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn"
Jul 20 16:54:27.849400 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.849378    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lwhk\" (UniqueName: \"kubernetes.io/projected/3e638f7d-358e-444f-87d4-dc7c40e95588-kube-api-access-5lwhk\") pod \"payload-processing-5bf47cd988-ffpqs\" (UID: \"3e638f7d-358e-444f-87d4-dc7c40e95588\") " pod="openshift-ingress/payload-processing-5bf47cd988-ffpqs"
Jul 20 16:54:27.950153 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.950123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwhzm\" (UniqueName: \"kubernetes.io/projected/dc696b70-0020-4372-baf8-0457fefc2593-kube-api-access-bwhzm\") pod \"payload-pre-processing-7677b774bb-9kjtn\" (UID: \"dc696b70-0020-4372-baf8-0457fefc2593\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn"
Jul 20 16:54:27.950330 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.950170    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e638f7d-358e-444f-87d4-dc7c40e95588-plugins-config-volume\") pod \"payload-processing-5bf47cd988-ffpqs\" (UID: \"3e638f7d-358e-444f-87d4-dc7c40e95588\") " pod="openshift-ingress/payload-processing-5bf47cd988-ffpqs"
Jul 20 16:54:27.950330 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.950193    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc696b70-0020-4372-baf8-0457fefc2593-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-9kjtn\" (UID: \"dc696b70-0020-4372-baf8-0457fefc2593\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn"
Jul 20 16:54:27.950330 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.950276    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lwhk\" (UniqueName: \"kubernetes.io/projected/3e638f7d-358e-444f-87d4-dc7c40e95588-kube-api-access-5lwhk\") pod \"payload-processing-5bf47cd988-ffpqs\" (UID: \"3e638f7d-358e-444f-87d4-dc7c40e95588\") " pod="openshift-ingress/payload-processing-5bf47cd988-ffpqs"
Jul 20 16:54:27.950863 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.950844    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e638f7d-358e-444f-87d4-dc7c40e95588-plugins-config-volume\") pod \"payload-processing-5bf47cd988-ffpqs\" (UID: \"3e638f7d-358e-444f-87d4-dc7c40e95588\") " pod="openshift-ingress/payload-processing-5bf47cd988-ffpqs"
Jul 20 16:54:27.950946 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.950898    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc696b70-0020-4372-baf8-0457fefc2593-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-9kjtn\" (UID: \"dc696b70-0020-4372-baf8-0457fefc2593\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn"
Jul 20 16:54:27.962812 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.962789    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwhzm\" (UniqueName: \"kubernetes.io/projected/dc696b70-0020-4372-baf8-0457fefc2593-kube-api-access-bwhzm\") pod \"payload-pre-processing-7677b774bb-9kjtn\" (UID: \"dc696b70-0020-4372-baf8-0457fefc2593\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn"
Jul 20 16:54:27.963035 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:27.962928    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lwhk\" (UniqueName: \"kubernetes.io/projected/3e638f7d-358e-444f-87d4-dc7c40e95588-kube-api-access-5lwhk\") pod \"payload-processing-5bf47cd988-ffpqs\" (UID: \"3e638f7d-358e-444f-87d4-dc7c40e95588\") " pod="openshift-ingress/payload-processing-5bf47cd988-ffpqs"
Jul 20 16:54:28.130353 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:28.130321    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn"
Jul 20 16:54:28.146073 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:28.146043    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-ffpqs"
Jul 20 16:54:28.244711 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:28.244666    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6" event={"ID":"2cde84b1-5fcf-47d5-8d7e-19a872432489","Type":"ContainerStarted","Data":"e9d0166a65f645535ef769220f7acb6e76515361dd3e4c43defd72eecee8d160"}
Jul 20 16:54:28.278076 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:28.278043    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-9kjtn"]
Jul 20 16:54:28.279735 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:54:28.279697    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc696b70_0020_4372_baf8_0457fefc2593.slice/crio-7e55d717ae9958cb5f41fa0564cd768c87a287eaa0043b1650993c34706243af WatchSource:0}: Error finding container 7e55d717ae9958cb5f41fa0564cd768c87a287eaa0043b1650993c34706243af: Status 404 returned error can't find the container with id 7e55d717ae9958cb5f41fa0564cd768c87a287eaa0043b1650993c34706243af
Jul 20 16:54:28.302346 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:28.302275    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-ffpqs"]
Jul 20 16:54:28.304585 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:54:28.304555    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e638f7d_358e_444f_87d4_dc7c40e95588.slice/crio-c0316c3023b8b8b3e4315a970f898e99ec0856df743ecc2e2f579281803764bc WatchSource:0}: Error finding container c0316c3023b8b8b3e4315a970f898e99ec0856df743ecc2e2f579281803764bc: Status 404 returned error can't find the container with id c0316c3023b8b8b3e4315a970f898e99ec0856df743ecc2e2f579281803764bc
Jul 20 16:54:29.253719 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:29.253681    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-ffpqs" event={"ID":"3e638f7d-358e-444f-87d4-dc7c40e95588","Type":"ContainerStarted","Data":"c0316c3023b8b8b3e4315a970f898e99ec0856df743ecc2e2f579281803764bc"}
Jul 20 16:54:29.255174 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:29.255144    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn" event={"ID":"dc696b70-0020-4372-baf8-0457fefc2593","Type":"ContainerStarted","Data":"7e55d717ae9958cb5f41fa0564cd768c87a287eaa0043b1650993c34706243af"}
Jul 20 16:54:30.263463 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:30.263424    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6" event={"ID":"2cde84b1-5fcf-47d5-8d7e-19a872432489","Type":"ContainerStarted","Data":"748f368fda54b4bb225ca3caf469894f4d4a8df29c778c85fa1f879162aa0318"}
Jul 20 16:54:30.263919 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:30.263720    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"
Jul 20 16:54:30.281440 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:30.281375    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6" podStartSLOduration=1.3649475500000001 podStartE2EDuration="3.28135395s" totalImagesPullingTime="1.9164064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:54:30.279830795 +0000 UTC m=+549.827686862" watchObservedRunningTime="2026-07-20 16:54:30.28135395 +0000 UTC m=+549.829210017"
Jul 20 16:54:31.270444 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:31.270405    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-ffpqs" event={"ID":"3e638f7d-358e-444f-87d4-dc7c40e95588","Type":"ContainerStarted","Data":"e71c30ccc54bb487d2744090f6e4fe104095fdf1ba1715b0727908862d01a795"}
Jul 20 16:54:31.270898 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:31.270504    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-ffpqs"
Jul 20 16:54:31.272019 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:31.271992    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn" event={"ID":"dc696b70-0020-4372-baf8-0457fefc2593","Type":"ContainerStarted","Data":"37894dc61be544cf0bfe54322a1dd3637f628bb663ae79c85686c8a27cd27ec9"}
Jul 20 16:54:31.288812 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:31.288757    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-ffpqs" podStartSLOduration=1.778891939 podStartE2EDuration="4.288735907s" totalImagesPullingTime="2.509843968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:54:31.285468409 +0000 UTC m=+550.833324476" watchObservedRunningTime="2026-07-20 16:54:31.288735907 +0000 UTC m=+550.836591974"
Jul 20 16:54:31.300481 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:31.300431    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn" podStartSLOduration=1.7603916069999999 podStartE2EDuration="4.300412434s" totalImagesPullingTime="2.540020827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:54:31.299326918 +0000 UTC m=+550.847182977" watchObservedRunningTime="2026-07-20 16:54:31.300412434 +0000 UTC m=+550.848268500"
Jul 20 16:54:32.276000 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:32.275967    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn"
Jul 20 16:54:36.277360 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:36.277334    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5df756f6bf-tfpb6"
Jul 20 16:54:37.318265 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:37.318215    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:37.586846 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:37.586748    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:42.277680 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:42.277649    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-ffpqs"
Jul 20 16:54:43.281827 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:43.281788    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn" podUID="dc696b70-0020-4372-baf8-0457fefc2593" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.38:9004: connect: connection refused"
Jul 20 16:54:46.814410 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:46.814372    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:46.925495 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:46.925458    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:53.280567 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:54:53.280539    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-9kjtn"
Jul 20 16:55:00.023808 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.023771    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"]
Jul 20 16:55:00.027380 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.027355    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.030036 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.030015    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 20 16:55:00.031097 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.031080    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 20 16:55:00.031196 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.031100    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k6shf\""
Jul 20 16:55:00.031196 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.031143    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 20 16:55:00.042795 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.042771    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"]
Jul 20 16:55:00.111485 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.111456    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:55:00.130835 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.130801    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v75p2\" (UniqueName: \"kubernetes.io/projected/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-kube-api-access-v75p2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.130998 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.130849    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.130998 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.130872    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.130998 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.130893    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.131115 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.130995    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.131115 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.131031    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.231991 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.231903    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v75p2\" (UniqueName: \"kubernetes.io/projected/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-kube-api-access-v75p2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.231991 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.231968    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.231991 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.231992    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.232322 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.232024    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.232322 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.232125    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.232322 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.232155    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.232492 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.232444    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.232637 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.232609    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.232637 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.232635    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.234496 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.234475    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.234783 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.234766    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.239858 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.239839    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v75p2\" (UniqueName: \"kubernetes.io/projected/a89f74e2-6a6a-4dec-a25d-50a41f3b0f33-kube-api-access-v75p2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f\" (UID: \"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.337450 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.337421    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:00.475043 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:00.475021    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"]
Jul 20 16:55:00.477520 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:55:00.477492    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda89f74e2_6a6a_4dec_a25d_50a41f3b0f33.slice/crio-3f362b06364e6404e32fc6591dcc9dc5c9cbc0c5912e10ad00d2a1a0f1bbb85a WatchSource:0}: Error finding container 3f362b06364e6404e32fc6591dcc9dc5c9cbc0c5912e10ad00d2a1a0f1bbb85a: Status 404 returned error can't find the container with id 3f362b06364e6404e32fc6591dcc9dc5c9cbc0c5912e10ad00d2a1a0f1bbb85a
Jul 20 16:55:01.410676 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:01.410637    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f" event={"ID":"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33","Type":"ContainerStarted","Data":"3f362b06364e6404e32fc6591dcc9dc5c9cbc0c5912e10ad00d2a1a0f1bbb85a"}
Jul 20 16:55:02.417348 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:02.417309    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f" event={"ID":"a89f74e2-6a6a-4dec-a25d-50a41f3b0f33","Type":"ContainerStarted","Data":"e2cb4acf7d22cabb062edd0a17a7bddf2efdd75bbf48e7e9d2e3bfd10eb273c2"}
Jul 20 16:55:02.436768 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:02.436720    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f" podStartSLOduration=1.2869503660000001 podStartE2EDuration="2.436704268s" totalImagesPullingTime="1.149753902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:55:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:55:02.435922187 +0000 UTC m=+581.983778252" watchObservedRunningTime="2026-07-20 16:55:02.436704268 +0000 UTC m=+581.984560335"
Jul 20 16:55:10.337685 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:10.337651    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:10.338276 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:10.337710    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:10.354641 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:10.354621    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:10.464751 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:10.464725    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f"
Jul 20 16:55:14.894509 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:14.894474    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:55:20.812728 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:20.812694    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 16:55:20.815488 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:20.815466    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 16:55:20.955045 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:20.955011    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:55:27.992181 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:27.992143    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"]
Jul 20 16:55:27.995425 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:27.995401    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:27.998285 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:27.998224    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 20 16:55:28.005622 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.005598    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"]
Jul 20 16:55:28.021983 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.021951    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:55:28.064057 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.064023    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kbv2\" (UniqueName: \"kubernetes.io/projected/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-kube-api-access-7kbv2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.064057 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.064061    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.064328 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.064089    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.064328 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.064163    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.064328 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.064193    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.064328 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.064212    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.164987 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.164958    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.165175 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.164999    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.165175 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.165031    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.165175 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.165142    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kbv2\" (UniqueName: \"kubernetes.io/projected/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-kube-api-access-7kbv2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.165175 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.165172    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.165417 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.165206    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.165491 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.165435    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.165573 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.165556    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.165625 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.165555    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.167418 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.167391    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.167942 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.167921    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.174908 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.174887    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kbv2\" (UniqueName: \"kubernetes.io/projected/6e66bf16-461e-4af6-995d-7fbff1dd5cf6-kube-api-access-7kbv2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjrfg\" (UID: \"6e66bf16-461e-4af6-995d-7fbff1dd5cf6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.308046 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.307954    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:28.449940 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.449912    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"]
Jul 20 16:55:28.451875 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:55:28.451852    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e66bf16_461e_4af6_995d_7fbff1dd5cf6.slice/crio-8fb0a6cb89f3565f52e8c90b7ed25cbd1b3b19009f63b853cbe15b64937d9369 WatchSource:0}: Error finding container 8fb0a6cb89f3565f52e8c90b7ed25cbd1b3b19009f63b853cbe15b64937d9369: Status 404 returned error can't find the container with id 8fb0a6cb89f3565f52e8c90b7ed25cbd1b3b19009f63b853cbe15b64937d9369
Jul 20 16:55:28.453597 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.453577    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 16:55:28.534423 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:28.534395    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg" event={"ID":"6e66bf16-461e-4af6-995d-7fbff1dd5cf6","Type":"ContainerStarted","Data":"8fb0a6cb89f3565f52e8c90b7ed25cbd1b3b19009f63b853cbe15b64937d9369"}
Jul 20 16:55:29.541608 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:29.541568    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg" event={"ID":"6e66bf16-461e-4af6-995d-7fbff1dd5cf6","Type":"ContainerStarted","Data":"d5e06bd83ed3d30ca8cfbcdf5400003379d1e304a315bfe97add5a7f07faa267"}
Jul 20 16:55:29.568973 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:29.568921    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg" podStartSLOduration=2.364155503 podStartE2EDuration="2.568901609s" totalImagesPullingTime="204.746106ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:55:29.567863158 +0000 UTC m=+609.115719225" watchObservedRunningTime="2026-07-20 16:55:29.568901609 +0000 UTC m=+609.116757674"
Jul 20 16:55:30.802199 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:30.802164    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:55:35.531334 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:35.531292    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:55:38.130733 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:38.130694    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:55:38.308114 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:38.308077    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:38.308341 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:38.308148    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:38.320965 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:38.320932    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:38.593526 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:38.593442    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjrfg"
Jul 20 16:55:43.518132 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.518049    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"]
Jul 20 16:55:43.523162 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.523142    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.528457 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.528442    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 20 16:55:43.534486 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.534469    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"]
Jul 20 16:55:43.597625 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.597587    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dea395ba-58f4-4e60-91b6-c85002ff099f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.597818 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.597676    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l9dz\" (UniqueName: \"kubernetes.io/projected/dea395ba-58f4-4e60-91b6-c85002ff099f-kube-api-access-5l9dz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.597818 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.597740    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dea395ba-58f4-4e60-91b6-c85002ff099f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.597818 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.597762    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dea395ba-58f4-4e60-91b6-c85002ff099f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.597818 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.597784    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dea395ba-58f4-4e60-91b6-c85002ff099f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.598032 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.597847    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dea395ba-58f4-4e60-91b6-c85002ff099f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.600715 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.600689    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:55:43.698762 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.698723    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dea395ba-58f4-4e60-91b6-c85002ff099f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.698762 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.698765    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dea395ba-58f4-4e60-91b6-c85002ff099f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.698967 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.698787    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dea395ba-58f4-4e60-91b6-c85002ff099f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.698967 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.698820    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dea395ba-58f4-4e60-91b6-c85002ff099f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.698967 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.698874    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dea395ba-58f4-4e60-91b6-c85002ff099f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.698967 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.698942    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l9dz\" (UniqueName: \"kubernetes.io/projected/dea395ba-58f4-4e60-91b6-c85002ff099f-kube-api-access-5l9dz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.699231 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.699207    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dea395ba-58f4-4e60-91b6-c85002ff099f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.699405 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.699342    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dea395ba-58f4-4e60-91b6-c85002ff099f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.699405 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.699235    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dea395ba-58f4-4e60-91b6-c85002ff099f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.701191 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.701166    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dea395ba-58f4-4e60-91b6-c85002ff099f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.701463 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.701448    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dea395ba-58f4-4e60-91b6-c85002ff099f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.707696 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.707673    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l9dz\" (UniqueName: \"kubernetes.io/projected/dea395ba-58f4-4e60-91b6-c85002ff099f-kube-api-access-5l9dz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh\" (UID: \"dea395ba-58f4-4e60-91b6-c85002ff099f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.832418 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.832337    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:43.975095 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:43.975062    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"]
Jul 20 16:55:43.976353 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:55:43.976325    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddea395ba_58f4_4e60_91b6_c85002ff099f.slice/crio-e0316967b7db915e50154cfdb5bf337b0f6162072c680eb59cdbf7c490791a53 WatchSource:0}: Error finding container e0316967b7db915e50154cfdb5bf337b0f6162072c680eb59cdbf7c490791a53: Status 404 returned error can't find the container with id e0316967b7db915e50154cfdb5bf337b0f6162072c680eb59cdbf7c490791a53
Jul 20 16:55:44.611877 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:44.611789    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh" event={"ID":"dea395ba-58f4-4e60-91b6-c85002ff099f","Type":"ContainerStarted","Data":"8a389560c5b3959be0529be44a9e1ee3705c439b4246859abf5da3dfdf832844"}
Jul 20 16:55:44.611877 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:44.611827    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh" event={"ID":"dea395ba-58f4-4e60-91b6-c85002ff099f","Type":"ContainerStarted","Data":"e0316967b7db915e50154cfdb5bf337b0f6162072c680eb59cdbf7c490791a53"}
Jul 20 16:55:44.631043 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:44.630989    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh" podStartSLOduration=1.322204675 podStartE2EDuration="1.630972281s" totalImagesPullingTime="308.767606ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:55:44.629819211 +0000 UTC m=+624.177675310" watchObservedRunningTime="2026-07-20 16:55:44.630972281 +0000 UTC m=+624.178828346"
Jul 20 16:55:53.832882 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:53.832849    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:53.832882 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:53.832894    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:53.848020 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:53.847995    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:55:54.670083 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:55:54.670057    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh"
Jul 20 16:56:36.500374 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:56:36.500318    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:57:20.333975 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:57:20.333898    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:57:20.437964 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:57:20.437927    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:57:21.493757 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:57:21.493717    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:57:21.592071 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:57:21.592040    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:07.418224 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:07.418182    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-qw2qm"]
Jul 20 16:58:07.418703 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:07.418389    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-77b8bc97bf-qw2qm" podUID="4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0" containerName="manager" containerID="cri-o://b459c6b4f604325553ded319d1fad846600a2b397561679ed92c8dd47b77e16e" gracePeriod=30
Jul 20 16:58:07.667593 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:07.667571    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77b8bc97bf-qw2qm"
Jul 20 16:58:07.736503 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:07.736432    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0-webhook-cert\") pod \"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0\" (UID: \"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0\") "
Jul 20 16:58:07.736503 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:07.736475    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvwbs\" (UniqueName: \"kubernetes.io/projected/4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0-kube-api-access-vvwbs\") pod \"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0\" (UID: \"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0\") "
Jul 20 16:58:07.738564 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:07.738535    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0" (UID: "4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:58:07.738662 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:07.738642    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0-kube-api-access-vvwbs" (OuterVolumeSpecName: "kube-api-access-vvwbs") pod "4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0" (UID: "4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0"). InnerVolumeSpecName "kube-api-access-vvwbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:58:07.837500 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:07.837469    2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0-webhook-cert\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:58:07.837500 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:07.837496    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvwbs\" (UniqueName: \"kubernetes.io/projected/4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0-kube-api-access-vvwbs\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:58:08.287255 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:08.287207    2568 generic.go:356] "Generic (PLEG): container finished" podID="4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0" containerID="b459c6b4f604325553ded319d1fad846600a2b397561679ed92c8dd47b77e16e" exitCode=0
Jul 20 16:58:08.287438 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:08.287271    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77b8bc97bf-qw2qm"
Jul 20 16:58:08.287438 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:08.287293    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77b8bc97bf-qw2qm" event={"ID":"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0","Type":"ContainerDied","Data":"b459c6b4f604325553ded319d1fad846600a2b397561679ed92c8dd47b77e16e"}
Jul 20 16:58:08.287438 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:08.287345    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77b8bc97bf-qw2qm" event={"ID":"4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0","Type":"ContainerDied","Data":"1cb04456ac9dd0921132ed643816ca4c985112c20cad624555a75b73960681d1"}
Jul 20 16:58:08.287438 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:08.287368    2568 scope.go:117] "RemoveContainer" containerID="b459c6b4f604325553ded319d1fad846600a2b397561679ed92c8dd47b77e16e"
Jul 20 16:58:08.297037 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:08.297024    2568 scope.go:117] "RemoveContainer" containerID="b459c6b4f604325553ded319d1fad846600a2b397561679ed92c8dd47b77e16e"
Jul 20 16:58:08.297319 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:58:08.297300    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b459c6b4f604325553ded319d1fad846600a2b397561679ed92c8dd47b77e16e\": container with ID starting with b459c6b4f604325553ded319d1fad846600a2b397561679ed92c8dd47b77e16e not found: ID does not exist" containerID="b459c6b4f604325553ded319d1fad846600a2b397561679ed92c8dd47b77e16e"
Jul 20 16:58:08.297366 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:08.297330    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b459c6b4f604325553ded319d1fad846600a2b397561679ed92c8dd47b77e16e"} err="failed to get container status \"b459c6b4f604325553ded319d1fad846600a2b397561679ed92c8dd47b77e16e\": rpc error: code = NotFound desc = could not find container \"b459c6b4f604325553ded319d1fad846600a2b397561679ed92c8dd47b77e16e\": container with ID starting with b459c6b4f604325553ded319d1fad846600a2b397561679ed92c8dd47b77e16e not found: ID does not exist"
Jul 20 16:58:08.309274 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:08.309233    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-qw2qm"]
Jul 20 16:58:08.313615 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:08.313593    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-qw2qm"]
Jul 20 16:58:08.895589 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:58:08.895557    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0" path="/var/lib/kubelet/pods/4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0/volumes"
Jul 20 16:59:00.429027 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:00.428993    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:09.117777 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.117741    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-kdlhm"]
Jul 20 16:59:09.118221 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.118123    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0" containerName="manager"
Jul 20 16:59:09.118221 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.118134    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0" containerName="manager"
Jul 20 16:59:09.118221 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.118194    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f0bb7bd-17e6-4c04-ad35-dca6063ce8c0" containerName="manager"
Jul 20 16:59:09.121174 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.121154    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77b8bc97bf-kdlhm"
Jul 20 16:59:09.123774 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.123753    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 16:59:09.123890 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.123755    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 16:59:09.125042 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.125023    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7lpl2\""
Jul 20 16:59:09.129320 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.128942    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-kdlhm"]
Jul 20 16:59:09.151468 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.151447    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps4pb\" (UniqueName: \"kubernetes.io/projected/e97c7c0f-efbf-48d4-bd26-6ca370201e4c-kube-api-access-ps4pb\") pod \"maas-controller-77b8bc97bf-kdlhm\" (UID: \"e97c7c0f-efbf-48d4-bd26-6ca370201e4c\") " pod="opendatahub/maas-controller-77b8bc97bf-kdlhm"
Jul 20 16:59:09.151576 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.151483    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e97c7c0f-efbf-48d4-bd26-6ca370201e4c-webhook-cert\") pod \"maas-controller-77b8bc97bf-kdlhm\" (UID: \"e97c7c0f-efbf-48d4-bd26-6ca370201e4c\") " pod="opendatahub/maas-controller-77b8bc97bf-kdlhm"
Jul 20 16:59:09.252190 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.252165    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps4pb\" (UniqueName: \"kubernetes.io/projected/e97c7c0f-efbf-48d4-bd26-6ca370201e4c-kube-api-access-ps4pb\") pod \"maas-controller-77b8bc97bf-kdlhm\" (UID: \"e97c7c0f-efbf-48d4-bd26-6ca370201e4c\") " pod="opendatahub/maas-controller-77b8bc97bf-kdlhm"
Jul 20 16:59:09.252354 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.252201    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e97c7c0f-efbf-48d4-bd26-6ca370201e4c-webhook-cert\") pod \"maas-controller-77b8bc97bf-kdlhm\" (UID: \"e97c7c0f-efbf-48d4-bd26-6ca370201e4c\") " pod="opendatahub/maas-controller-77b8bc97bf-kdlhm"
Jul 20 16:59:09.254753 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.254722    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e97c7c0f-efbf-48d4-bd26-6ca370201e4c-webhook-cert\") pod \"maas-controller-77b8bc97bf-kdlhm\" (UID: \"e97c7c0f-efbf-48d4-bd26-6ca370201e4c\") " pod="opendatahub/maas-controller-77b8bc97bf-kdlhm"
Jul 20 16:59:09.261283 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.261264    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps4pb\" (UniqueName: \"kubernetes.io/projected/e97c7c0f-efbf-48d4-bd26-6ca370201e4c-kube-api-access-ps4pb\") pod \"maas-controller-77b8bc97bf-kdlhm\" (UID: \"e97c7c0f-efbf-48d4-bd26-6ca370201e4c\") " pod="opendatahub/maas-controller-77b8bc97bf-kdlhm"
Jul 20 16:59:09.432732 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.432695    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77b8bc97bf-kdlhm"
Jul 20 16:59:09.572482 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:09.572453    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-kdlhm"]
Jul 20 16:59:09.573542 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:59:09.573519    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode97c7c0f_efbf_48d4_bd26_6ca370201e4c.slice/crio-3cc959473ee5bb4141f09b77c698f7ea7061c23dcc20e656336d699e21239aa1 WatchSource:0}: Error finding container 3cc959473ee5bb4141f09b77c698f7ea7061c23dcc20e656336d699e21239aa1: Status 404 returned error can't find the container with id 3cc959473ee5bb4141f09b77c698f7ea7061c23dcc20e656336d699e21239aa1
Jul 20 16:59:10.578384 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:10.578351    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77b8bc97bf-kdlhm" event={"ID":"e97c7c0f-efbf-48d4-bd26-6ca370201e4c","Type":"ContainerStarted","Data":"e459ebe8b14d3338213c5d61e059567870935798e4576522069efffed08467b2"}
Jul 20 16:59:10.578814 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:10.578433    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77b8bc97bf-kdlhm" event={"ID":"e97c7c0f-efbf-48d4-bd26-6ca370201e4c","Type":"ContainerStarted","Data":"3cc959473ee5bb4141f09b77c698f7ea7061c23dcc20e656336d699e21239aa1"}
Jul 20 16:59:10.578814 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:10.578460    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77b8bc97bf-kdlhm"
Jul 20 16:59:10.595076 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:10.595030    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77b8bc97bf-kdlhm" podStartSLOduration=1.1701028199999999 podStartE2EDuration="1.595015765s" totalImagesPullingTime="424.912945ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:59:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:59:10.593411033 +0000 UTC m=+830.141267100" watchObservedRunningTime="2026-07-20 16:59:10.595015765 +0000 UTC m=+830.142871862"
Jul 20 16:59:21.585972 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:21.585943    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77b8bc97bf-kdlhm"
Jul 20 16:59:26.126723 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.126688    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:26.239300 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.239269    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-kdlhm"]
Jul 20 16:59:26.239485 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.239468    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-77b8bc97bf-kdlhm" podUID="e97c7c0f-efbf-48d4-bd26-6ca370201e4c" containerName="manager" containerID="cri-o://e459ebe8b14d3338213c5d61e059567870935798e4576522069efffed08467b2" gracePeriod=30
Jul 20 16:59:26.494076 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.494052    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77b8bc97bf-kdlhm"
Jul 20 16:59:26.603181 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.603145    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps4pb\" (UniqueName: \"kubernetes.io/projected/e97c7c0f-efbf-48d4-bd26-6ca370201e4c-kube-api-access-ps4pb\") pod \"e97c7c0f-efbf-48d4-bd26-6ca370201e4c\" (UID: \"e97c7c0f-efbf-48d4-bd26-6ca370201e4c\") "
Jul 20 16:59:26.603376 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.603210    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e97c7c0f-efbf-48d4-bd26-6ca370201e4c-webhook-cert\") pod \"e97c7c0f-efbf-48d4-bd26-6ca370201e4c\" (UID: \"e97c7c0f-efbf-48d4-bd26-6ca370201e4c\") "
Jul 20 16:59:26.605463 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.605432    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e97c7c0f-efbf-48d4-bd26-6ca370201e4c-kube-api-access-ps4pb" (OuterVolumeSpecName: "kube-api-access-ps4pb") pod "e97c7c0f-efbf-48d4-bd26-6ca370201e4c" (UID: "e97c7c0f-efbf-48d4-bd26-6ca370201e4c"). InnerVolumeSpecName "kube-api-access-ps4pb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:59:26.605575 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.605462    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e97c7c0f-efbf-48d4-bd26-6ca370201e4c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e97c7c0f-efbf-48d4-bd26-6ca370201e4c" (UID: "e97c7c0f-efbf-48d4-bd26-6ca370201e4c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:59:26.655426 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.655401    2568 generic.go:356] "Generic (PLEG): container finished" podID="e97c7c0f-efbf-48d4-bd26-6ca370201e4c" containerID="e459ebe8b14d3338213c5d61e059567870935798e4576522069efffed08467b2" exitCode=0
Jul 20 16:59:26.655555 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.655446    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77b8bc97bf-kdlhm"
Jul 20 16:59:26.655555 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.655466    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77b8bc97bf-kdlhm" event={"ID":"e97c7c0f-efbf-48d4-bd26-6ca370201e4c","Type":"ContainerDied","Data":"e459ebe8b14d3338213c5d61e059567870935798e4576522069efffed08467b2"}
Jul 20 16:59:26.655555 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.655509    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77b8bc97bf-kdlhm" event={"ID":"e97c7c0f-efbf-48d4-bd26-6ca370201e4c","Type":"ContainerDied","Data":"3cc959473ee5bb4141f09b77c698f7ea7061c23dcc20e656336d699e21239aa1"}
Jul 20 16:59:26.655555 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.655530    2568 scope.go:117] "RemoveContainer" containerID="e459ebe8b14d3338213c5d61e059567870935798e4576522069efffed08467b2"
Jul 20 16:59:26.664708 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.664694    2568 scope.go:117] "RemoveContainer" containerID="e459ebe8b14d3338213c5d61e059567870935798e4576522069efffed08467b2"
Jul 20 16:59:26.664961 ip-10-0-128-8 kubenswrapper[2568]: E0720 16:59:26.664940    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e459ebe8b14d3338213c5d61e059567870935798e4576522069efffed08467b2\": container with ID starting with e459ebe8b14d3338213c5d61e059567870935798e4576522069efffed08467b2 not found: ID does not exist" containerID="e459ebe8b14d3338213c5d61e059567870935798e4576522069efffed08467b2"
Jul 20 16:59:26.665018 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.664977    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e459ebe8b14d3338213c5d61e059567870935798e4576522069efffed08467b2"} err="failed to get container status \"e459ebe8b14d3338213c5d61e059567870935798e4576522069efffed08467b2\": rpc error: code = NotFound desc = could not find container \"e459ebe8b14d3338213c5d61e059567870935798e4576522069efffed08467b2\": container with ID starting with e459ebe8b14d3338213c5d61e059567870935798e4576522069efffed08467b2 not found: ID does not exist"
Jul 20 16:59:26.678198 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.678179    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-kdlhm"]
Jul 20 16:59:26.682563 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.682545    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-kdlhm"]
Jul 20 16:59:26.704144 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.704126    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps4pb\" (UniqueName: \"kubernetes.io/projected/e97c7c0f-efbf-48d4-bd26-6ca370201e4c-kube-api-access-ps4pb\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:59:26.704144 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.704146    2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e97c7c0f-efbf-48d4-bd26-6ca370201e4c-webhook-cert\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 16:59:26.901226 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:26.901191    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e97c7c0f-efbf-48d4-bd26-6ca370201e4c" path="/var/lib/kubelet/pods/e97c7c0f-efbf-48d4-bd26-6ca370201e4c/volumes"
Jul 20 16:59:27.362773 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.362745    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:27.451305 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.451269    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-bvk88"]
Jul 20 16:59:27.451628 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.451617    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e97c7c0f-efbf-48d4-bd26-6ca370201e4c" containerName="manager"
Jul 20 16:59:27.451672 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.451631    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e97c7c0f-efbf-48d4-bd26-6ca370201e4c" containerName="manager"
Jul 20 16:59:27.451707 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.451697    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e97c7c0f-efbf-48d4-bd26-6ca370201e4c" containerName="manager"
Jul 20 16:59:27.456009 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.455990    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77b8bc97bf-bvk88"
Jul 20 16:59:27.459796 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.459776    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 20 16:59:27.459920 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.459813    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 20 16:59:27.459920 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.459813    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7lpl2\""
Jul 20 16:59:27.466674 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.466652    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-bvk88"]
Jul 20 16:59:27.510593 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.510563    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5wkp\" (UniqueName: \"kubernetes.io/projected/2cfb5b69-20eb-4e61-a1a3-0b7cc362a952-kube-api-access-g5wkp\") pod \"maas-controller-77b8bc97bf-bvk88\" (UID: \"2cfb5b69-20eb-4e61-a1a3-0b7cc362a952\") " pod="opendatahub/maas-controller-77b8bc97bf-bvk88"
Jul 20 16:59:27.510593 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.510594    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cfb5b69-20eb-4e61-a1a3-0b7cc362a952-webhook-cert\") pod \"maas-controller-77b8bc97bf-bvk88\" (UID: \"2cfb5b69-20eb-4e61-a1a3-0b7cc362a952\") " pod="opendatahub/maas-controller-77b8bc97bf-bvk88"
Jul 20 16:59:27.611393 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.611362    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5wkp\" (UniqueName: \"kubernetes.io/projected/2cfb5b69-20eb-4e61-a1a3-0b7cc362a952-kube-api-access-g5wkp\") pod \"maas-controller-77b8bc97bf-bvk88\" (UID: \"2cfb5b69-20eb-4e61-a1a3-0b7cc362a952\") " pod="opendatahub/maas-controller-77b8bc97bf-bvk88"
Jul 20 16:59:27.611393 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.611394    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cfb5b69-20eb-4e61-a1a3-0b7cc362a952-webhook-cert\") pod \"maas-controller-77b8bc97bf-bvk88\" (UID: \"2cfb5b69-20eb-4e61-a1a3-0b7cc362a952\") " pod="opendatahub/maas-controller-77b8bc97bf-bvk88"
Jul 20 16:59:27.613905 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.613851    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cfb5b69-20eb-4e61-a1a3-0b7cc362a952-webhook-cert\") pod \"maas-controller-77b8bc97bf-bvk88\" (UID: \"2cfb5b69-20eb-4e61-a1a3-0b7cc362a952\") " pod="opendatahub/maas-controller-77b8bc97bf-bvk88"
Jul 20 16:59:27.620027 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.620004    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5wkp\" (UniqueName: \"kubernetes.io/projected/2cfb5b69-20eb-4e61-a1a3-0b7cc362a952-kube-api-access-g5wkp\") pod \"maas-controller-77b8bc97bf-bvk88\" (UID: \"2cfb5b69-20eb-4e61-a1a3-0b7cc362a952\") " pod="opendatahub/maas-controller-77b8bc97bf-bvk88"
Jul 20 16:59:27.767033 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.767005    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77b8bc97bf-bvk88"
Jul 20 16:59:27.908338 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:27.908302    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77b8bc97bf-bvk88"]
Jul 20 16:59:27.909591 ip-10-0-128-8 kubenswrapper[2568]: W0720 16:59:27.909567    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cfb5b69_20eb_4e61_a1a3_0b7cc362a952.slice/crio-3b7081ef609af6684c6b9a58176e6335ce9b48338f21f01ac88cb308cc9cabbc WatchSource:0}: Error finding container 3b7081ef609af6684c6b9a58176e6335ce9b48338f21f01ac88cb308cc9cabbc: Status 404 returned error can't find the container with id 3b7081ef609af6684c6b9a58176e6335ce9b48338f21f01ac88cb308cc9cabbc
Jul 20 16:59:28.667296 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:28.667233    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77b8bc97bf-bvk88" event={"ID":"2cfb5b69-20eb-4e61-a1a3-0b7cc362a952","Type":"ContainerStarted","Data":"845c3a84fb21cf98485d145d2c1c47c27be9f9ea90daf6e7766480d2390de05e"}
Jul 20 16:59:28.667663 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:28.667320    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77b8bc97bf-bvk88" event={"ID":"2cfb5b69-20eb-4e61-a1a3-0b7cc362a952","Type":"ContainerStarted","Data":"3b7081ef609af6684c6b9a58176e6335ce9b48338f21f01ac88cb308cc9cabbc"}
Jul 20 16:59:28.667663 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:28.667463    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77b8bc97bf-bvk88"
Jul 20 16:59:39.674976 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:39.674945    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77b8bc97bf-bvk88"
Jul 20 16:59:39.693306 ip-10-0-128-8 kubenswrapper[2568]: I0720 16:59:39.693234    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77b8bc97bf-bvk88" podStartSLOduration=12.136835843 podStartE2EDuration="12.6932181s" totalImagesPullingTime="556.382257ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:59:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:59:28.709681249 +0000 UTC m=+848.257537315" watchObservedRunningTime="2026-07-20 16:59:39.6932181 +0000 UTC m=+859.241074165"
Jul 20 17:00:05.132095 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:00:05.132063    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:05.229037 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:00:05.229009    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:05.343622 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:00:05.343583    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:11.174797 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:00:11.174757    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:11.306860 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:00:11.306830    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:11.407320 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:00:11.407292    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:11.506984 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:00:11.506907    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:17.073450 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:00:17.073414    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:17.148979 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:00:17.148946    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:20.842734 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:00:20.842704    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:00:20.843840 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:00:20.843822    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:02:09.397161 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:02:09.397123    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:12.511897 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:02:12.511839    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:26.698478 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:02:26.698445    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:04:09.179575 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:09.179544    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"]
Jul 20 17:04:09.180133 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:09.179773    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n" podUID="aa6fda9f-5098-4028-b6ec-61a9795d3667" containerName="manager" containerID="cri-o://06990e9f72151fb3b49d2f77ce1439e0c025997a644be3779389ed6ad7b49f61" gracePeriod=10
Jul 20 17:04:11.838630 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:11.838606    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"
Jul 20 17:04:11.913419 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:11.913389    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppzfb\" (UniqueName: \"kubernetes.io/projected/aa6fda9f-5098-4028-b6ec-61a9795d3667-kube-api-access-ppzfb\") pod \"aa6fda9f-5098-4028-b6ec-61a9795d3667\" (UID: \"aa6fda9f-5098-4028-b6ec-61a9795d3667\") "
Jul 20 17:04:11.913584 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:11.913469    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa6fda9f-5098-4028-b6ec-61a9795d3667-extensions-socket-volume\") pod \"aa6fda9f-5098-4028-b6ec-61a9795d3667\" (UID: \"aa6fda9f-5098-4028-b6ec-61a9795d3667\") "
Jul 20 17:04:11.913820 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:11.913800    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6fda9f-5098-4028-b6ec-61a9795d3667-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "aa6fda9f-5098-4028-b6ec-61a9795d3667" (UID: "aa6fda9f-5098-4028-b6ec-61a9795d3667"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:04:11.915590 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:11.915564    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa6fda9f-5098-4028-b6ec-61a9795d3667-kube-api-access-ppzfb" (OuterVolumeSpecName: "kube-api-access-ppzfb") pod "aa6fda9f-5098-4028-b6ec-61a9795d3667" (UID: "aa6fda9f-5098-4028-b6ec-61a9795d3667"). InnerVolumeSpecName "kube-api-access-ppzfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:04:12.014573 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:12.014476    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppzfb\" (UniqueName: \"kubernetes.io/projected/aa6fda9f-5098-4028-b6ec-61a9795d3667-kube-api-access-ppzfb\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:04:12.014573 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:12.014512    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa6fda9f-5098-4028-b6ec-61a9795d3667-extensions-socket-volume\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:04:12.023172 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:12.023141    2568 generic.go:356] "Generic (PLEG): container finished" podID="aa6fda9f-5098-4028-b6ec-61a9795d3667" containerID="06990e9f72151fb3b49d2f77ce1439e0c025997a644be3779389ed6ad7b49f61" exitCode=0
Jul 20 17:04:12.023336 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:12.023186    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"
Jul 20 17:04:12.023336 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:12.023230    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n" event={"ID":"aa6fda9f-5098-4028-b6ec-61a9795d3667","Type":"ContainerDied","Data":"06990e9f72151fb3b49d2f77ce1439e0c025997a644be3779389ed6ad7b49f61"}
Jul 20 17:04:12.023336 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:12.023314    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n" event={"ID":"aa6fda9f-5098-4028-b6ec-61a9795d3667","Type":"ContainerDied","Data":"e6b225bafea9799736f942fcfa6c38c74e436307766ae5566aa77e11b7d10abb"}
Jul 20 17:04:12.023336 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:12.023340    2568 scope.go:117] "RemoveContainer" containerID="06990e9f72151fb3b49d2f77ce1439e0c025997a644be3779389ed6ad7b49f61"
Jul 20 17:04:12.041172 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:12.041149    2568 scope.go:117] "RemoveContainer" containerID="06990e9f72151fb3b49d2f77ce1439e0c025997a644be3779389ed6ad7b49f61"
Jul 20 17:04:12.041499 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:04:12.041465    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06990e9f72151fb3b49d2f77ce1439e0c025997a644be3779389ed6ad7b49f61\": container with ID starting with 06990e9f72151fb3b49d2f77ce1439e0c025997a644be3779389ed6ad7b49f61 not found: ID does not exist" containerID="06990e9f72151fb3b49d2f77ce1439e0c025997a644be3779389ed6ad7b49f61"
Jul 20 17:04:12.041606 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:12.041504    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06990e9f72151fb3b49d2f77ce1439e0c025997a644be3779389ed6ad7b49f61"} err="failed to get container status \"06990e9f72151fb3b49d2f77ce1439e0c025997a644be3779389ed6ad7b49f61\": rpc error: code = NotFound desc = could not find container \"06990e9f72151fb3b49d2f77ce1439e0c025997a644be3779389ed6ad7b49f61\": container with ID starting with 06990e9f72151fb3b49d2f77ce1439e0c025997a644be3779389ed6ad7b49f61 not found: ID does not exist"
Jul 20 17:04:12.049413 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:12.049384    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"]
Jul 20 17:04:12.053750 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:12.053727    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ct47n"]
Jul 20 17:04:12.896614 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:12.896578    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa6fda9f-5098-4028-b6ec-61a9795d3667" path="/var/lib/kubelet/pods/aa6fda9f-5098-4028-b6ec-61a9795d3667/volumes"
Jul 20 17:04:26.107645 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:04:26.107613    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:08.059530 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:08.059498    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:10.362482 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.362449    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8"]
Jul 20 17:05:10.362960 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.362821    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa6fda9f-5098-4028-b6ec-61a9795d3667" containerName="manager"
Jul 20 17:05:10.362960 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.362831    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa6fda9f-5098-4028-b6ec-61a9795d3667" containerName="manager"
Jul 20 17:05:10.362960 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.362903    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa6fda9f-5098-4028-b6ec-61a9795d3667" containerName="manager"
Jul 20 17:05:10.365854 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.365838    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8"
Jul 20 17:05:10.369912 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.369894    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-l28jj\""
Jul 20 17:05:10.383550 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.383528    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8"]
Jul 20 17:05:10.409534 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.409499    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/387dab05-928e-40ef-8944-8f58b98e4f4c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sqdq8\" (UID: \"387dab05-928e-40ef-8944-8f58b98e4f4c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8"
Jul 20 17:05:10.409710 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.409569    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9t8v\" (UniqueName: \"kubernetes.io/projected/387dab05-928e-40ef-8944-8f58b98e4f4c-kube-api-access-c9t8v\") pod \"kuadrant-operator-controller-manager-86878c769f-sqdq8\" (UID: \"387dab05-928e-40ef-8944-8f58b98e4f4c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8"
Jul 20 17:05:10.510307 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.510235    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9t8v\" (UniqueName: \"kubernetes.io/projected/387dab05-928e-40ef-8944-8f58b98e4f4c-kube-api-access-c9t8v\") pod \"kuadrant-operator-controller-manager-86878c769f-sqdq8\" (UID: \"387dab05-928e-40ef-8944-8f58b98e4f4c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8"
Jul 20 17:05:10.510490 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.510350    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/387dab05-928e-40ef-8944-8f58b98e4f4c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sqdq8\" (UID: \"387dab05-928e-40ef-8944-8f58b98e4f4c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8"
Jul 20 17:05:10.510706 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.510690    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/387dab05-928e-40ef-8944-8f58b98e4f4c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sqdq8\" (UID: \"387dab05-928e-40ef-8944-8f58b98e4f4c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8"
Jul 20 17:05:10.527369 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.527233    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9t8v\" (UniqueName: \"kubernetes.io/projected/387dab05-928e-40ef-8944-8f58b98e4f4c-kube-api-access-c9t8v\") pod \"kuadrant-operator-controller-manager-86878c769f-sqdq8\" (UID: \"387dab05-928e-40ef-8944-8f58b98e4f4c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8"
Jul 20 17:05:10.675418 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.675385    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8"
Jul 20 17:05:10.824064 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:10.824039    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8"]
Jul 20 17:05:10.825891 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:05:10.825860    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod387dab05_928e_40ef_8944_8f58b98e4f4c.slice/crio-4e28d0ebb124185e661b161455eabdbcc5085ae0fbf651f5ac08fa0e827f7076 WatchSource:0}: Error finding container 4e28d0ebb124185e661b161455eabdbcc5085ae0fbf651f5ac08fa0e827f7076: Status 404 returned error can't find the container with id 4e28d0ebb124185e661b161455eabdbcc5085ae0fbf651f5ac08fa0e827f7076
Jul 20 17:05:11.313788 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:11.313753    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8" event={"ID":"387dab05-928e-40ef-8944-8f58b98e4f4c","Type":"ContainerStarted","Data":"649ee22702460f4b2fc85255914ae07915ddb15ef6c86a4ae22de2a76a518e29"}
Jul 20 17:05:11.313991 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:11.313816    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8" event={"ID":"387dab05-928e-40ef-8944-8f58b98e4f4c","Type":"ContainerStarted","Data":"4e28d0ebb124185e661b161455eabdbcc5085ae0fbf651f5ac08fa0e827f7076"}
Jul 20 17:05:11.313991 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:11.313847    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8"
Jul 20 17:05:11.338904 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:11.338849    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8" podStartSLOduration=1.3388306079999999 podStartE2EDuration="1.338830608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:05:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:05:11.3362988 +0000 UTC m=+1190.884154865" watchObservedRunningTime="2026-07-20 17:05:11.338830608 +0000 UTC m=+1190.886686674"
Jul 20 17:05:20.877995 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:20.877966    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:05:20.878524 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:20.878390    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:05:22.321586 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:05:22.321555    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sqdq8"
Jul 20 17:07:46.720921 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.720839    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"]
Jul 20 17:07:46.724153 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.724135    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.726897 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.726878    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-ef1427a3-kserve-self-signed-certs\""
Jul 20 17:07:46.741827 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.741801    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"]
Jul 20 17:07:46.828829 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.828791    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:07:46.852972 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.852935    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-model-cache\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.853160 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.852987    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvqsq\" (UniqueName: \"kubernetes.io/projected/314f27e6-d6e6-495d-9411-0fe81c3d6f52-kube-api-access-wvqsq\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.853160 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.853034    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-tmp-dir\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.853160 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.853112    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-dshm\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.853360 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.853204    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/314f27e6-d6e6-495d-9411-0fe81c3d6f52-tls-certs\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.853360 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.853280    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-home\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.953695 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.953661    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/314f27e6-d6e6-495d-9411-0fe81c3d6f52-tls-certs\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.953695 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.953699    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-home\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.953910 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.953723    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-model-cache\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.953910 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.953874    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvqsq\" (UniqueName: \"kubernetes.io/projected/314f27e6-d6e6-495d-9411-0fe81c3d6f52-kube-api-access-wvqsq\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.954027 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.953908    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-tmp-dir\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.954207 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.954112    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-home\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.954207 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.954156    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-model-cache\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.954207 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.954178    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-dshm\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.954207 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.954189    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-tmp-dir\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.957589 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.957463    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/314f27e6-d6e6-495d-9411-0fe81c3d6f52-tls-certs\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.965110 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.963485    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvqsq\" (UniqueName: \"kubernetes.io/projected/314f27e6-d6e6-495d-9411-0fe81c3d6f52-kube-api-access-wvqsq\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:46.965928 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:46.965907    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-dshm\") pod \"e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") " pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:47.032971 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:47.032884    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:47.190663 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:47.190636    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"]
Jul 20 17:07:47.193390 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:07:47.193360    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod314f27e6_d6e6_495d_9411_0fe81c3d6f52.slice/crio-efc5128cffbec6746780bed3debacd2d4be02e0e2d986d082135b5c6535aac67 WatchSource:0}: Error finding container efc5128cffbec6746780bed3debacd2d4be02e0e2d986d082135b5c6535aac67: Status 404 returned error can't find the container with id efc5128cffbec6746780bed3debacd2d4be02e0e2d986d082135b5c6535aac67
Jul 20 17:07:48.050434 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:48.050394    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9" event={"ID":"314f27e6-d6e6-495d-9411-0fe81c3d6f52","Type":"ContainerStarted","Data":"a95e0ece96a1a526d05864dfb60796f9d05275d06b87fb3cebabd0b8ef124d60"}
Jul 20 17:07:48.050900 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:48.050451    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9" event={"ID":"314f27e6-d6e6-495d-9411-0fe81c3d6f52","Type":"ContainerStarted","Data":"efc5128cffbec6746780bed3debacd2d4be02e0e2d986d082135b5c6535aac67"}
Jul 20 17:07:48.069946 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:48.069901    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9" podStartSLOduration=2.069886151 podStartE2EDuration="2.069886151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:07:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:07:48.068488774 +0000 UTC m=+1347.616344840" watchObservedRunningTime="2026-07-20 17:07:48.069886151 +0000 UTC m=+1347.617742217"
Jul 20 17:07:53.481783 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.481748    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"]
Jul 20 17:07:53.489592 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.489564    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.493315 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.493287    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-ef1427a3-kserve-self-signed-certs\""
Jul 20 17:07:53.502705 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.502675    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"]
Jul 20 17:07:53.578071 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.578039    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:07:53.613382 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.613354    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-home\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.613554 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.613395    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-model-cache\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.613554 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.613462    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zh7s\" (UniqueName: \"kubernetes.io/projected/bfc0882d-4e9f-4292-9772-24ced9893682-kube-api-access-9zh7s\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.613554 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.613497    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-dshm\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.613667 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.613548    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-tmp-dir\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.613667 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.613600    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfc0882d-4e9f-4292-9772-24ced9893682-tls-certs\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.714099 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.714060    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-dshm\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.714099 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.714098    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-tmp-dir\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.714388 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.714123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfc0882d-4e9f-4292-9772-24ced9893682-tls-certs\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.714388 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.714177    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-home\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.714388 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.714215    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-model-cache\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.714388 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.714289    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zh7s\" (UniqueName: \"kubernetes.io/projected/bfc0882d-4e9f-4292-9772-24ced9893682-kube-api-access-9zh7s\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.714628 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.714605    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-tmp-dir\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.714693 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.714678    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-home\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.714743 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.714694    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-model-cache\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.716636 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.716607    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-dshm\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.717278 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.717226    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfc0882d-4e9f-4292-9772-24ced9893682-tls-certs\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.722381 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.722361    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zh7s\" (UniqueName: \"kubernetes.io/projected/bfc0882d-4e9f-4292-9772-24ced9893682-kube-api-access-9zh7s\") pod \"e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") " pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.801805 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.801718    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:07:53.945501 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:53.944973    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"]
Jul 20 17:07:53.947665 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:07:53.947640    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfc0882d_4e9f_4292_9772_24ced9893682.slice/crio-2194abefcab50e64d5b13cdfee782676608f0b08a8c56f22c486f54a53c3bff6 WatchSource:0}: Error finding container 2194abefcab50e64d5b13cdfee782676608f0b08a8c56f22c486f54a53c3bff6: Status 404 returned error can't find the container with id 2194abefcab50e64d5b13cdfee782676608f0b08a8c56f22c486f54a53c3bff6
Jul 20 17:07:54.082366 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:54.082281    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g" event={"ID":"bfc0882d-4e9f-4292-9772-24ced9893682","Type":"ContainerStarted","Data":"d683b468f7c76dbb85dea1b36128c43c17220c3452cf778e2ee49014bf266553"}
Jul 20 17:07:54.082366 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:54.082331    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g" event={"ID":"bfc0882d-4e9f-4292-9772-24ced9893682","Type":"ContainerStarted","Data":"2194abefcab50e64d5b13cdfee782676608f0b08a8c56f22c486f54a53c3bff6"}
Jul 20 17:07:57.033490 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:57.033390    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:57.033922 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:57.033551    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:57.046280 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:57.046255    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:07:57.064833 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:57.064788    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g" podStartSLOduration=4.064773097 podStartE2EDuration="4.064773097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:07:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:07:54.101442481 +0000 UTC m=+1353.649298548" watchObservedRunningTime="2026-07-20 17:07:57.064773097 +0000 UTC m=+1356.612629159"
Jul 20 17:07:57.108968 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:07:57.108939    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:08:03.802453 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:08:03.802409    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:08:03.802848 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:08:03.802554    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:08:03.815334 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:08:03.815307    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:08:04.142858 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:08:04.142825    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:08:44.682715 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:08:44.682676    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"]
Jul 20 17:08:44.706650 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:08:44.706615    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:45.696305 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:08:45.696273    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"]
Jul 20 17:08:45.718586 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:08:45.718558    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:55.835722 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:08:55.835685    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:59.703481 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:08:59.703441    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9" podUID="314f27e6-d6e6-495d-9411-0fe81c3d6f52" containerName="main" containerID="cri-o://a95e0ece96a1a526d05864dfb60796f9d05275d06b87fb3cebabd0b8ef124d60" gracePeriod=45
Jul 20 17:08:59.957726 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:08:59.957664    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:09:00.112823 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.112795    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvqsq\" (UniqueName: \"kubernetes.io/projected/314f27e6-d6e6-495d-9411-0fe81c3d6f52-kube-api-access-wvqsq\") pod \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") "
Jul 20 17:09:00.113004 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.112857    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/314f27e6-d6e6-495d-9411-0fe81c3d6f52-tls-certs\") pod \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") "
Jul 20 17:09:00.113004 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.112885    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-dshm\") pod \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") "
Jul 20 17:09:00.113004 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.112907    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-model-cache\") pod \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") "
Jul 20 17:09:00.113004 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.112956    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-home\") pod \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") "
Jul 20 17:09:00.113004 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.112994    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-tmp-dir\") pod \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\" (UID: \"314f27e6-d6e6-495d-9411-0fe81c3d6f52\") "
Jul 20 17:09:00.113279 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.113191    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-home" (OuterVolumeSpecName: "home") pod "314f27e6-d6e6-495d-9411-0fe81c3d6f52" (UID: "314f27e6-d6e6-495d-9411-0fe81c3d6f52"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:09:00.113279 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.113204    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-model-cache" (OuterVolumeSpecName: "model-cache") pod "314f27e6-d6e6-495d-9411-0fe81c3d6f52" (UID: "314f27e6-d6e6-495d-9411-0fe81c3d6f52"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:09:00.113393 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.113325    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "314f27e6-d6e6-495d-9411-0fe81c3d6f52" (UID: "314f27e6-d6e6-495d-9411-0fe81c3d6f52"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:09:00.115342 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.115322    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-dshm" (OuterVolumeSpecName: "dshm") pod "314f27e6-d6e6-495d-9411-0fe81c3d6f52" (UID: "314f27e6-d6e6-495d-9411-0fe81c3d6f52"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:09:00.115596 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.115578    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/314f27e6-d6e6-495d-9411-0fe81c3d6f52-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "314f27e6-d6e6-495d-9411-0fe81c3d6f52" (UID: "314f27e6-d6e6-495d-9411-0fe81c3d6f52"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:09:00.115651 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.115634    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/314f27e6-d6e6-495d-9411-0fe81c3d6f52-kube-api-access-wvqsq" (OuterVolumeSpecName: "kube-api-access-wvqsq") pod "314f27e6-d6e6-495d-9411-0fe81c3d6f52" (UID: "314f27e6-d6e6-495d-9411-0fe81c3d6f52"). InnerVolumeSpecName "kube-api-access-wvqsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:09:00.214661 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.214582    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/314f27e6-d6e6-495d-9411-0fe81c3d6f52-tls-certs\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:09:00.214661 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.214612    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-dshm\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:09:00.214661 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.214620    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-model-cache\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:09:00.214661 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.214629    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-home\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:09:00.214661 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.214637    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/314f27e6-d6e6-495d-9411-0fe81c3d6f52-tmp-dir\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:09:00.214661 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.214645    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvqsq\" (UniqueName: \"kubernetes.io/projected/314f27e6-d6e6-495d-9411-0fe81c3d6f52-kube-api-access-wvqsq\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:09:00.403133 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.403098    2568 generic.go:356] "Generic (PLEG): container finished" podID="314f27e6-d6e6-495d-9411-0fe81c3d6f52" containerID="a95e0ece96a1a526d05864dfb60796f9d05275d06b87fb3cebabd0b8ef124d60" exitCode=0
Jul 20 17:09:00.403133 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.403135    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"
Jul 20 17:09:00.403358 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.403182    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9" event={"ID":"314f27e6-d6e6-495d-9411-0fe81c3d6f52","Type":"ContainerDied","Data":"a95e0ece96a1a526d05864dfb60796f9d05275d06b87fb3cebabd0b8ef124d60"}
Jul 20 17:09:00.403358 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.403228    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9" event={"ID":"314f27e6-d6e6-495d-9411-0fe81c3d6f52","Type":"ContainerDied","Data":"efc5128cffbec6746780bed3debacd2d4be02e0e2d986d082135b5c6535aac67"}
Jul 20 17:09:00.403358 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.403268    2568 scope.go:117] "RemoveContainer" containerID="a95e0ece96a1a526d05864dfb60796f9d05275d06b87fb3cebabd0b8ef124d60"
Jul 20 17:09:00.412677 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.412656    2568 scope.go:117] "RemoveContainer" containerID="a95e0ece96a1a526d05864dfb60796f9d05275d06b87fb3cebabd0b8ef124d60"
Jul 20 17:09:00.412961 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:09:00.412943    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a95e0ece96a1a526d05864dfb60796f9d05275d06b87fb3cebabd0b8ef124d60\": container with ID starting with a95e0ece96a1a526d05864dfb60796f9d05275d06b87fb3cebabd0b8ef124d60 not found: ID does not exist" containerID="a95e0ece96a1a526d05864dfb60796f9d05275d06b87fb3cebabd0b8ef124d60"
Jul 20 17:09:00.413037 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.412968    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a95e0ece96a1a526d05864dfb60796f9d05275d06b87fb3cebabd0b8ef124d60"} err="failed to get container status \"a95e0ece96a1a526d05864dfb60796f9d05275d06b87fb3cebabd0b8ef124d60\": rpc error: code = NotFound desc = could not find container \"a95e0ece96a1a526d05864dfb60796f9d05275d06b87fb3cebabd0b8ef124d60\": container with ID starting with a95e0ece96a1a526d05864dfb60796f9d05275d06b87fb3cebabd0b8ef124d60 not found: ID does not exist"
Jul 20 17:09:00.425323 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.425298    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"]
Jul 20 17:09:00.429864 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.429844    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-ef1427a3-kserve-6944ffb7f5-rcbt9"]
Jul 20 17:09:00.717386 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.717334    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g" podUID="bfc0882d-4e9f-4292-9772-24ced9893682" containerName="main" containerID="cri-o://d683b468f7c76dbb85dea1b36128c43c17220c3452cf778e2ee49014bf266553" gracePeriod=45
Jul 20 17:09:00.904021 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.903918    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="314f27e6-d6e6-495d-9411-0fe81c3d6f52" path="/var/lib/kubelet/pods/314f27e6-d6e6-495d-9411-0fe81c3d6f52/volumes"
Jul 20 17:09:00.977357 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:00.977293    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:09:01.122722 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.122689    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zh7s\" (UniqueName: \"kubernetes.io/projected/bfc0882d-4e9f-4292-9772-24ced9893682-kube-api-access-9zh7s\") pod \"bfc0882d-4e9f-4292-9772-24ced9893682\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") "
Jul 20 17:09:01.122722 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.122727    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-tmp-dir\") pod \"bfc0882d-4e9f-4292-9772-24ced9893682\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") "
Jul 20 17:09:01.122976 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.122793    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfc0882d-4e9f-4292-9772-24ced9893682-tls-certs\") pod \"bfc0882d-4e9f-4292-9772-24ced9893682\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") "
Jul 20 17:09:01.122976 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.122815    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-home\") pod \"bfc0882d-4e9f-4292-9772-24ced9893682\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") "
Jul 20 17:09:01.122976 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.122838    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-model-cache\") pod \"bfc0882d-4e9f-4292-9772-24ced9893682\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") "
Jul 20 17:09:01.122976 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.122862    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-dshm\") pod \"bfc0882d-4e9f-4292-9772-24ced9893682\" (UID: \"bfc0882d-4e9f-4292-9772-24ced9893682\") "
Jul 20 17:09:01.123182 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.123081    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-home" (OuterVolumeSpecName: "home") pod "bfc0882d-4e9f-4292-9772-24ced9893682" (UID: "bfc0882d-4e9f-4292-9772-24ced9893682"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:09:01.123182 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.123108    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bfc0882d-4e9f-4292-9772-24ced9893682" (UID: "bfc0882d-4e9f-4292-9772-24ced9893682"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:09:01.123182 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.123172    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-model-cache" (OuterVolumeSpecName: "model-cache") pod "bfc0882d-4e9f-4292-9772-24ced9893682" (UID: "bfc0882d-4e9f-4292-9772-24ced9893682"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:09:01.125055 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.125030    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfc0882d-4e9f-4292-9772-24ced9893682-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bfc0882d-4e9f-4292-9772-24ced9893682" (UID: "bfc0882d-4e9f-4292-9772-24ced9893682"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:09:01.125055 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.125042    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfc0882d-4e9f-4292-9772-24ced9893682-kube-api-access-9zh7s" (OuterVolumeSpecName: "kube-api-access-9zh7s") pod "bfc0882d-4e9f-4292-9772-24ced9893682" (UID: "bfc0882d-4e9f-4292-9772-24ced9893682"). InnerVolumeSpecName "kube-api-access-9zh7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:09:01.125220 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.125121    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-dshm" (OuterVolumeSpecName: "dshm") pod "bfc0882d-4e9f-4292-9772-24ced9893682" (UID: "bfc0882d-4e9f-4292-9772-24ced9893682"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:09:01.223929 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.223897    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfc0882d-4e9f-4292-9772-24ced9893682-tls-certs\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:09:01.223929 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.223923    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-home\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:09:01.224136 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.223952    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-model-cache\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:09:01.224136 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.223962    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-dshm\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:09:01.224136 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.223970    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zh7s\" (UniqueName: \"kubernetes.io/projected/bfc0882d-4e9f-4292-9772-24ced9893682-kube-api-access-9zh7s\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:09:01.224136 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.223980    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfc0882d-4e9f-4292-9772-24ced9893682-tmp-dir\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:09:01.409639 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.409603    2568 generic.go:356] "Generic (PLEG): container finished" podID="bfc0882d-4e9f-4292-9772-24ced9893682" containerID="d683b468f7c76dbb85dea1b36128c43c17220c3452cf778e2ee49014bf266553" exitCode=0
Jul 20 17:09:01.409826 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.409651    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"
Jul 20 17:09:01.409826 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.409678    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g" event={"ID":"bfc0882d-4e9f-4292-9772-24ced9893682","Type":"ContainerDied","Data":"d683b468f7c76dbb85dea1b36128c43c17220c3452cf778e2ee49014bf266553"}
Jul 20 17:09:01.409826 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.409719    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g" event={"ID":"bfc0882d-4e9f-4292-9772-24ced9893682","Type":"ContainerDied","Data":"2194abefcab50e64d5b13cdfee782676608f0b08a8c56f22c486f54a53c3bff6"}
Jul 20 17:09:01.409826 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.409736    2568 scope.go:117] "RemoveContainer" containerID="d683b468f7c76dbb85dea1b36128c43c17220c3452cf778e2ee49014bf266553"
Jul 20 17:09:01.419128 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.419112    2568 scope.go:117] "RemoveContainer" containerID="d683b468f7c76dbb85dea1b36128c43c17220c3452cf778e2ee49014bf266553"
Jul 20 17:09:01.419406 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:09:01.419388    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d683b468f7c76dbb85dea1b36128c43c17220c3452cf778e2ee49014bf266553\": container with ID starting with d683b468f7c76dbb85dea1b36128c43c17220c3452cf778e2ee49014bf266553 not found: ID does not exist" containerID="d683b468f7c76dbb85dea1b36128c43c17220c3452cf778e2ee49014bf266553"
Jul 20 17:09:01.419459 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.419420    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d683b468f7c76dbb85dea1b36128c43c17220c3452cf778e2ee49014bf266553"} err="failed to get container status \"d683b468f7c76dbb85dea1b36128c43c17220c3452cf778e2ee49014bf266553\": rpc error: code = NotFound desc = could not find container \"d683b468f7c76dbb85dea1b36128c43c17220c3452cf778e2ee49014bf266553\": container with ID starting with d683b468f7c76dbb85dea1b36128c43c17220c3452cf778e2ee49014bf266553 not found: ID does not exist"
Jul 20 17:09:01.431319 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.431292    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"]
Jul 20 17:09:01.435704 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:01.435679    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-ef1427a3-kserve-659b6486ff-h9s7g"]
Jul 20 17:09:02.896167 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:09:02.896135    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfc0882d-4e9f-4292-9772-24ced9893682" path="/var/lib/kubelet/pods/bfc0882d-4e9f-4292-9772-24ced9893682/volumes"
Jul 20 17:10:20.905859 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:10:20.905832    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:10:20.907003 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:10:20.906986    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:10:52.478145 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:10:52.478114    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:52.568214 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:10:52.568175    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:11:29.056288 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:11:29.056231    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:11:54.326102 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:11:54.326066    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:11:58.302910 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:11:58.302878    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:11:58.358172 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:11:58.358148    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:11:59.652946 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:11:59.652904    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:07.913414 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:07.913378    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:13.298584 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:13.298497    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:16.264374 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:16.264338    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:16.366955 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:16.366913    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:16.716708 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:16.716670    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:22.878753 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:22.878716    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:23.025491 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:23.025456    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:23.177566 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:23.177441    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:23.474489 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:23.474393    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:23.937515 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:23.937484    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:26.225096 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.225051    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:26.312138 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.312101    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"]
Jul 20 17:12:26.312508 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.312495    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfc0882d-4e9f-4292-9772-24ced9893682" containerName="main"
Jul 20 17:12:26.312567 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.312509    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfc0882d-4e9f-4292-9772-24ced9893682" containerName="main"
Jul 20 17:12:26.312567 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.312526    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="314f27e6-d6e6-495d-9411-0fe81c3d6f52" containerName="main"
Jul 20 17:12:26.312567 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.312532    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="314f27e6-d6e6-495d-9411-0fe81c3d6f52" containerName="main"
Jul 20 17:12:26.312681 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.312582    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfc0882d-4e9f-4292-9772-24ced9893682" containerName="main"
Jul 20 17:12:26.312681 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.312592    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="314f27e6-d6e6-495d-9411-0fe81c3d6f52" containerName="main"
Jul 20 17:12:26.315082 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.315066    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:12:26.318932 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.318911    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-82838e1d-serving-cert\""
Jul 20 17:12:26.336172 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.336142    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"]
Jul 20 17:12:26.342669 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.342637    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:26.437153 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.437062    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:26.442632 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.442600    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dc684aa-861a-4738-bd30-999e73780db1-maas-api-tls\") pod \"maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l\" (UID: \"5dc684aa-861a-4738-bd30-999e73780db1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:12:26.442632 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.442632    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc684aa-861a-4738-bd30-999e73780db1-openshift-service-ca\") pod \"maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l\" (UID: \"5dc684aa-861a-4738-bd30-999e73780db1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:12:26.442852 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.442663    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw77n\" (UniqueName: \"kubernetes.io/projected/5dc684aa-861a-4738-bd30-999e73780db1-kube-api-access-mw77n\") pod \"maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l\" (UID: \"5dc684aa-861a-4738-bd30-999e73780db1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:12:26.544015 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.543902    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dc684aa-861a-4738-bd30-999e73780db1-maas-api-tls\") pod \"maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l\" (UID: \"5dc684aa-861a-4738-bd30-999e73780db1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:12:26.544015 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.543955    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc684aa-861a-4738-bd30-999e73780db1-openshift-service-ca\") pod \"maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l\" (UID: \"5dc684aa-861a-4738-bd30-999e73780db1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:12:26.544015 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.543989    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw77n\" (UniqueName: \"kubernetes.io/projected/5dc684aa-861a-4738-bd30-999e73780db1-kube-api-access-mw77n\") pod \"maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l\" (UID: \"5dc684aa-861a-4738-bd30-999e73780db1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:12:26.544823 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.544799    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc684aa-861a-4738-bd30-999e73780db1-openshift-service-ca\") pod \"maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l\" (UID: \"5dc684aa-861a-4738-bd30-999e73780db1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:12:26.546577 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.546555    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dc684aa-861a-4738-bd30-999e73780db1-maas-api-tls\") pod \"maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l\" (UID: \"5dc684aa-861a-4738-bd30-999e73780db1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:12:26.558570 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.558541    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw77n\" (UniqueName: \"kubernetes.io/projected/5dc684aa-861a-4738-bd30-999e73780db1-kube-api-access-mw77n\") pod \"maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l\" (UID: \"5dc684aa-861a-4738-bd30-999e73780db1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:12:26.613381 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.613319    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:26.626229 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.626195    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:12:26.769620 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.769598    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"]
Jul 20 17:12:26.771578 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:12:26.771543    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc684aa_861a_4738_bd30_999e73780db1.slice/crio-1e80fcd763e29e3101471a4288a0742b73338f6483020a493234ca9576c90814 WatchSource:0}: Error finding container 1e80fcd763e29e3101471a4288a0742b73338f6483020a493234ca9576c90814: Status 404 returned error can't find the container with id 1e80fcd763e29e3101471a4288a0742b73338f6483020a493234ca9576c90814
Jul 20 17:12:26.772924 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.772905    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 17:12:26.948931 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:26.948893    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:27.055700 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:27.055660    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:27.381271 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:27.381229    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l" event={"ID":"5dc684aa-861a-4738-bd30-999e73780db1","Type":"ContainerStarted","Data":"f1828f3392144a84d02acfd0336a3fac87c1a4b88078ff4357ea59d95d439090"}
Jul 20 17:12:27.381636 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:27.381286    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l" event={"ID":"5dc684aa-861a-4738-bd30-999e73780db1","Type":"ContainerStarted","Data":"1e80fcd763e29e3101471a4288a0742b73338f6483020a493234ca9576c90814"}
Jul 20 17:12:27.381636 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:27.381326    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:12:27.426418 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:27.426370    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l" podStartSLOduration=1.005683455 podStartE2EDuration="1.426350966s" totalImagesPullingTime="420.667511ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:12:27.425835511 +0000 UTC m=+1626.973691577" watchObservedRunningTime="2026-07-20 17:12:27.426350966 +0000 UTC m=+1626.974207032"
Jul 20 17:12:33.391912 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:12:33.391879    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:13:02.294329 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.294291    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"]
Jul 20 17:13:02.294888 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.294556    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l" podUID="5dc684aa-861a-4738-bd30-999e73780db1" containerName="maas-api" containerID="cri-o://f1828f3392144a84d02acfd0336a3fac87c1a4b88078ff4357ea59d95d439090" gracePeriod=30
Jul 20 17:13:02.318439 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.318414    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:02.541632 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.541603    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:13:02.544409 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.544325    2568 generic.go:356] "Generic (PLEG): container finished" podID="5dc684aa-861a-4738-bd30-999e73780db1" containerID="f1828f3392144a84d02acfd0336a3fac87c1a4b88078ff4357ea59d95d439090" exitCode=0
Jul 20 17:13:02.544409 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.544371    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"
Jul 20 17:13:02.544409 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.544403    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l" event={"ID":"5dc684aa-861a-4738-bd30-999e73780db1","Type":"ContainerDied","Data":"f1828f3392144a84d02acfd0336a3fac87c1a4b88078ff4357ea59d95d439090"}
Jul 20 17:13:02.544667 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.544449    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l" event={"ID":"5dc684aa-861a-4738-bd30-999e73780db1","Type":"ContainerDied","Data":"1e80fcd763e29e3101471a4288a0742b73338f6483020a493234ca9576c90814"}
Jul 20 17:13:02.544667 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.544472    2568 scope.go:117] "RemoveContainer" containerID="f1828f3392144a84d02acfd0336a3fac87c1a4b88078ff4357ea59d95d439090"
Jul 20 17:13:02.553228 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.553208    2568 scope.go:117] "RemoveContainer" containerID="f1828f3392144a84d02acfd0336a3fac87c1a4b88078ff4357ea59d95d439090"
Jul 20 17:13:02.553540 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:13:02.553520    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1828f3392144a84d02acfd0336a3fac87c1a4b88078ff4357ea59d95d439090\": container with ID starting with f1828f3392144a84d02acfd0336a3fac87c1a4b88078ff4357ea59d95d439090 not found: ID does not exist" containerID="f1828f3392144a84d02acfd0336a3fac87c1a4b88078ff4357ea59d95d439090"
Jul 20 17:13:02.553610 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.553549    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1828f3392144a84d02acfd0336a3fac87c1a4b88078ff4357ea59d95d439090"} err="failed to get container status \"f1828f3392144a84d02acfd0336a3fac87c1a4b88078ff4357ea59d95d439090\": rpc error: code = NotFound desc = could not find container \"f1828f3392144a84d02acfd0336a3fac87c1a4b88078ff4357ea59d95d439090\": container with ID starting with f1828f3392144a84d02acfd0336a3fac87c1a4b88078ff4357ea59d95d439090 not found: ID does not exist"
Jul 20 17:13:02.673346 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.673318    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw77n\" (UniqueName: \"kubernetes.io/projected/5dc684aa-861a-4738-bd30-999e73780db1-kube-api-access-mw77n\") pod \"5dc684aa-861a-4738-bd30-999e73780db1\" (UID: \"5dc684aa-861a-4738-bd30-999e73780db1\") "
Jul 20 17:13:02.673531 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.673370    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dc684aa-861a-4738-bd30-999e73780db1-maas-api-tls\") pod \"5dc684aa-861a-4738-bd30-999e73780db1\" (UID: \"5dc684aa-861a-4738-bd30-999e73780db1\") "
Jul 20 17:13:02.673531 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.673451    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc684aa-861a-4738-bd30-999e73780db1-openshift-service-ca\") pod \"5dc684aa-861a-4738-bd30-999e73780db1\" (UID: \"5dc684aa-861a-4738-bd30-999e73780db1\") "
Jul 20 17:13:02.673909 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.673865    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dc684aa-861a-4738-bd30-999e73780db1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5dc684aa-861a-4738-bd30-999e73780db1" (UID: "5dc684aa-861a-4738-bd30-999e73780db1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:13:02.675762 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.675740    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dc684aa-861a-4738-bd30-999e73780db1-kube-api-access-mw77n" (OuterVolumeSpecName: "kube-api-access-mw77n") pod "5dc684aa-861a-4738-bd30-999e73780db1" (UID: "5dc684aa-861a-4738-bd30-999e73780db1"). InnerVolumeSpecName "kube-api-access-mw77n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:13:02.675845 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.675764    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dc684aa-861a-4738-bd30-999e73780db1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5dc684aa-861a-4738-bd30-999e73780db1" (UID: "5dc684aa-861a-4738-bd30-999e73780db1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:13:02.774549 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.774514    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc684aa-861a-4738-bd30-999e73780db1-openshift-service-ca\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:13:02.774549 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.774545    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw77n\" (UniqueName: \"kubernetes.io/projected/5dc684aa-861a-4738-bd30-999e73780db1-kube-api-access-mw77n\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:13:02.774549 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.774556    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dc684aa-861a-4738-bd30-999e73780db1-maas-api-tls\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:13:02.868467 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.868435    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"]
Jul 20 17:13:02.872294 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.872270    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82838e1d-fcc4bf97f-5wn7l"]
Jul 20 17:13:02.896818 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:02.896789    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dc684aa-861a-4738-bd30-999e73780db1" path="/var/lib/kubelet/pods/5dc684aa-861a-4738-bd30-999e73780db1/volumes"
Jul 20 17:13:06.962432 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:06.962385    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:07.172072 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:07.172043    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:07.335692 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:07.335596    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:07.646827 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:07.646790    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:07.972476 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:07.972391    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:08.192417 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:08.192383    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:08.366928 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:08.366806    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:08.494421 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:08.494384    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:14.260313 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.260273    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:14.494022 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.493990    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"]
Jul 20 17:13:14.494540 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.494521    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dc684aa-861a-4738-bd30-999e73780db1" containerName="maas-api"
Jul 20 17:13:14.494540 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.494541    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dc684aa-861a-4738-bd30-999e73780db1" containerName="maas-api"
Jul 20 17:13:14.494690 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.494607    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dc684aa-861a-4738-bd30-999e73780db1" containerName="maas-api"
Jul 20 17:13:14.498848 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.498829    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:14.501435 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.501416    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-df6eeabb-serving-cert\""
Jul 20 17:13:14.508120 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.508097    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"]
Jul 20 17:13:14.539126 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.539030    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:14.574952 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.574920    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j68hm\" (UniqueName: \"kubernetes.io/projected/5527d543-8740-46f2-a35d-ff6c75e6cdb1-kube-api-access-j68hm\") pod \"maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx\" (UID: \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:14.575115 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.574987    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5527d543-8740-46f2-a35d-ff6c75e6cdb1-openshift-service-ca\") pod \"maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx\" (UID: \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:14.575115 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.575024    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5527d543-8740-46f2-a35d-ff6c75e6cdb1-maas-api-tls\") pod \"maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx\" (UID: \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:14.675762 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.675731    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j68hm\" (UniqueName: \"kubernetes.io/projected/5527d543-8740-46f2-a35d-ff6c75e6cdb1-kube-api-access-j68hm\") pod \"maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx\" (UID: \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:14.675933 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.675827    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5527d543-8740-46f2-a35d-ff6c75e6cdb1-openshift-service-ca\") pod \"maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx\" (UID: \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:14.675933 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.675863    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5527d543-8740-46f2-a35d-ff6c75e6cdb1-maas-api-tls\") pod \"maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx\" (UID: \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:14.676025 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:13:14.675964    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-serving-cert: secret "maas-api-e2e-migrate-df6eeabb-serving-cert" not found
Jul 20 17:13:14.676061 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:13:14.676042    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5527d543-8740-46f2-a35d-ff6c75e6cdb1-maas-api-tls podName:5527d543-8740-46f2-a35d-ff6c75e6cdb1 nodeName:}" failed. No retries permitted until 2026-07-20 17:13:15.17602164 +0000 UTC m=+1674.723877688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5527d543-8740-46f2-a35d-ff6c75e6cdb1-maas-api-tls") pod "maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx" (UID: "5527d543-8740-46f2-a35d-ff6c75e6cdb1") : secret "maas-api-e2e-migrate-df6eeabb-serving-cert" not found
Jul 20 17:13:14.676638 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.676605    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5527d543-8740-46f2-a35d-ff6c75e6cdb1-openshift-service-ca\") pod \"maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx\" (UID: \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:14.685230 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.685205    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j68hm\" (UniqueName: \"kubernetes.io/projected/5527d543-8740-46f2-a35d-ff6c75e6cdb1-kube-api-access-j68hm\") pod \"maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx\" (UID: \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:14.787073 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.787038    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:14.993688 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:14.993631    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:15.095186 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:15.095146    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:15.178974 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:15.178899    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5527d543-8740-46f2-a35d-ff6c75e6cdb1-maas-api-tls\") pod \"maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx\" (UID: \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:15.181812 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:15.181783    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5527d543-8740-46f2-a35d-ff6c75e6cdb1-maas-api-tls\") pod \"maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx\" (UID: \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:15.411127 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:15.411094    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:15.561705 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:15.561656    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"]
Jul 20 17:13:15.565453 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:13:15.565423    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5527d543_8740_46f2_a35d_ff6c75e6cdb1.slice/crio-4dadadb267af1fb6a35e61075a97839cbb4a93be76ec23d00ef453644b7ff55f WatchSource:0}: Error finding container 4dadadb267af1fb6a35e61075a97839cbb4a93be76ec23d00ef453644b7ff55f: Status 404 returned error can't find the container with id 4dadadb267af1fb6a35e61075a97839cbb4a93be76ec23d00ef453644b7ff55f
Jul 20 17:13:15.608058 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:15.608024    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx" event={"ID":"5527d543-8740-46f2-a35d-ff6c75e6cdb1","Type":"ContainerStarted","Data":"4dadadb267af1fb6a35e61075a97839cbb4a93be76ec23d00ef453644b7ff55f"}
Jul 20 17:13:16.614593 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:16.614559    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx" event={"ID":"5527d543-8740-46f2-a35d-ff6c75e6cdb1","Type":"ContainerStarted","Data":"68d1e3a2b80f0594e2da7a151ec24125b67332d71cddef2fdaad6d0815d86af4"}
Jul 20 17:13:16.614999 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:16.614640    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:16.636288 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:16.636218    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx" podStartSLOduration=2.242601427 podStartE2EDuration="2.636201507s" totalImagesPullingTime="393.60008ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:13:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:13:16.633730353 +0000 UTC m=+1676.181586430" watchObservedRunningTime="2026-07-20 17:13:16.636201507 +0000 UTC m=+1676.184057572"
Jul 20 17:13:22.624673 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:22.624646    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:50.226038 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.226004    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"]
Jul 20 17:13:50.226672 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.226273    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx" podUID="5527d543-8740-46f2-a35d-ff6c75e6cdb1" containerName="maas-api" containerID="cri-o://68d1e3a2b80f0594e2da7a151ec24125b67332d71cddef2fdaad6d0815d86af4" gracePeriod=30
Jul 20 17:13:50.261441 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.261413    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:50.488655 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.488591    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:50.580813 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.580770    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5527d543-8740-46f2-a35d-ff6c75e6cdb1-openshift-service-ca\") pod \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\" (UID: \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\") "
Jul 20 17:13:50.580813 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.580812    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5527d543-8740-46f2-a35d-ff6c75e6cdb1-maas-api-tls\") pod \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\" (UID: \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\") "
Jul 20 17:13:50.581063 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.580850    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j68hm\" (UniqueName: \"kubernetes.io/projected/5527d543-8740-46f2-a35d-ff6c75e6cdb1-kube-api-access-j68hm\") pod \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\" (UID: \"5527d543-8740-46f2-a35d-ff6c75e6cdb1\") "
Jul 20 17:13:50.581233 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.581206    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5527d543-8740-46f2-a35d-ff6c75e6cdb1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5527d543-8740-46f2-a35d-ff6c75e6cdb1" (UID: "5527d543-8740-46f2-a35d-ff6c75e6cdb1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:13:50.583140 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.583118    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5527d543-8740-46f2-a35d-ff6c75e6cdb1-kube-api-access-j68hm" (OuterVolumeSpecName: "kube-api-access-j68hm") pod "5527d543-8740-46f2-a35d-ff6c75e6cdb1" (UID: "5527d543-8740-46f2-a35d-ff6c75e6cdb1"). InnerVolumeSpecName "kube-api-access-j68hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:13:50.583212 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.583142    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5527d543-8740-46f2-a35d-ff6c75e6cdb1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5527d543-8740-46f2-a35d-ff6c75e6cdb1" (UID: "5527d543-8740-46f2-a35d-ff6c75e6cdb1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:13:50.682228 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.682179    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5527d543-8740-46f2-a35d-ff6c75e6cdb1-openshift-service-ca\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:13:50.682228 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.682222    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5527d543-8740-46f2-a35d-ff6c75e6cdb1-maas-api-tls\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:13:50.682228 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.682235    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j68hm\" (UniqueName: \"kubernetes.io/projected/5527d543-8740-46f2-a35d-ff6c75e6cdb1-kube-api-access-j68hm\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:13:50.776633 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.776529    2568 generic.go:356] "Generic (PLEG): container finished" podID="5527d543-8740-46f2-a35d-ff6c75e6cdb1" containerID="68d1e3a2b80f0594e2da7a151ec24125b67332d71cddef2fdaad6d0815d86af4" exitCode=0
Jul 20 17:13:50.776633 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.776574    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"
Jul 20 17:13:50.776633 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.776569    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx" event={"ID":"5527d543-8740-46f2-a35d-ff6c75e6cdb1","Type":"ContainerDied","Data":"68d1e3a2b80f0594e2da7a151ec24125b67332d71cddef2fdaad6d0815d86af4"}
Jul 20 17:13:50.776911 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.776672    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx" event={"ID":"5527d543-8740-46f2-a35d-ff6c75e6cdb1","Type":"ContainerDied","Data":"4dadadb267af1fb6a35e61075a97839cbb4a93be76ec23d00ef453644b7ff55f"}
Jul 20 17:13:50.776911 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.776705    2568 scope.go:117] "RemoveContainer" containerID="68d1e3a2b80f0594e2da7a151ec24125b67332d71cddef2fdaad6d0815d86af4"
Jul 20 17:13:50.785765 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.785748    2568 scope.go:117] "RemoveContainer" containerID="68d1e3a2b80f0594e2da7a151ec24125b67332d71cddef2fdaad6d0815d86af4"
Jul 20 17:13:50.786025 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:13:50.786010    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68d1e3a2b80f0594e2da7a151ec24125b67332d71cddef2fdaad6d0815d86af4\": container with ID starting with 68d1e3a2b80f0594e2da7a151ec24125b67332d71cddef2fdaad6d0815d86af4 not found: ID does not exist" containerID="68d1e3a2b80f0594e2da7a151ec24125b67332d71cddef2fdaad6d0815d86af4"
Jul 20 17:13:50.786067 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.786032    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68d1e3a2b80f0594e2da7a151ec24125b67332d71cddef2fdaad6d0815d86af4"} err="failed to get container status \"68d1e3a2b80f0594e2da7a151ec24125b67332d71cddef2fdaad6d0815d86af4\": rpc error: code = NotFound desc = could not find container \"68d1e3a2b80f0594e2da7a151ec24125b67332d71cddef2fdaad6d0815d86af4\": container with ID starting with 68d1e3a2b80f0594e2da7a151ec24125b67332d71cddef2fdaad6d0815d86af4 not found: ID does not exist"
Jul 20 17:13:50.798697 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.798673    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"]
Jul 20 17:13:50.800469 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.800450    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-df6eeabb-5d7566b744-cqpmx"]
Jul 20 17:13:50.896385 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:50.896352    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5527d543-8740-46f2-a35d-ff6c75e6cdb1" path="/var/lib/kubelet/pods/5527d543-8740-46f2-a35d-ff6c75e6cdb1/volumes"
Jul 20 17:13:54.538124 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:54.538077    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:54.699646 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:54.699614    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:54.962959 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:54.962931    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:55.206785 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:55.206754    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:55.577335 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:55.577283    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:56.022563 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:56.022530    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:57.727636 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:57.727607    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:57.885908 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:57.885877    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"]
Jul 20 17:13:57.886262 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:57.886226    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5527d543-8740-46f2-a35d-ff6c75e6cdb1" containerName="maas-api"
Jul 20 17:13:57.886262 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:57.886256    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5527d543-8740-46f2-a35d-ff6c75e6cdb1" containerName="maas-api"
Jul 20 17:13:57.886371 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:57.886354    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5527d543-8740-46f2-a35d-ff6c75e6cdb1" containerName="maas-api"
Jul 20 17:13:57.889638 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:57.889618    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:13:57.892695 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:57.892672    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:57.893489 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:57.893474    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e9002fce-serving-cert\""
Jul 20 17:13:57.920623 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:57.920598    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"]
Jul 20 17:13:58.042764 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.042676    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhgbs\" (UniqueName: \"kubernetes.io/projected/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-kube-api-access-bhgbs\") pod \"maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n\" (UID: \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:13:58.042764 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.042713    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-maas-api-tls\") pod \"maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n\" (UID: \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:13:58.042977 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.042824    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-openshift-service-ca\") pod \"maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n\" (UID: \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:13:58.096397 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.096358    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:58.143763 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.143731    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-openshift-service-ca\") pod \"maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n\" (UID: \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:13:58.143913 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.143776    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhgbs\" (UniqueName: \"kubernetes.io/projected/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-kube-api-access-bhgbs\") pod \"maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n\" (UID: \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:13:58.143913 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.143798    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-maas-api-tls\") pod \"maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n\" (UID: \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:13:58.144633 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.144559    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-openshift-service-ca\") pod \"maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n\" (UID: \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:13:58.146333 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.146314    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-maas-api-tls\") pod \"maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n\" (UID: \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:13:58.163367 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.163344    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhgbs\" (UniqueName: \"kubernetes.io/projected/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-kube-api-access-bhgbs\") pod \"maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n\" (UID: \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:13:58.199836 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.199816    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:13:58.345190 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.345150    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"]
Jul 20 17:13:58.346204 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:13:58.346172    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode66d3516_f1eb_4a2d_9e1e_4ccb816faa43.slice/crio-a801112d17879b6e9a731656decbab3aee7ebacec8b8644c1733879183aeeedd WatchSource:0}: Error finding container a801112d17879b6e9a731656decbab3aee7ebacec8b8644c1733879183aeeedd: Status 404 returned error can't find the container with id a801112d17879b6e9a731656decbab3aee7ebacec8b8644c1733879183aeeedd
Jul 20 17:13:58.786609 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.786580    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:58.819376 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.819343    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n" event={"ID":"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43","Type":"ContainerStarted","Data":"a801112d17879b6e9a731656decbab3aee7ebacec8b8644c1733879183aeeedd"}
Jul 20 17:13:58.924790 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:58.924762    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:59.825924 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:59.825888    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n" event={"ID":"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43","Type":"ContainerStarted","Data":"158575e99fb5d640fb007b1ce8d9e30d33aa0c33123f178323bda67ac482c9db"}
Jul 20 17:13:59.826483 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:59.825995    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:13:59.848387 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:13:59.848341    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n" podStartSLOduration=2.265897433 podStartE2EDuration="2.84832464s" totalImagesPullingTime="582.427207ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:13:59.845875179 +0000 UTC m=+1719.393731282" watchObservedRunningTime="2026-07-20 17:13:59.84832464 +0000 UTC m=+1719.396180706"
Jul 20 17:14:05.837323 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:05.837294    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:14:34.102751 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.102714    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"]
Jul 20 17:14:34.103165 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.102933    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n" podUID="e66d3516-f1eb-4a2d-9e1e-4ccb816faa43" containerName="maas-api" containerID="cri-o://158575e99fb5d640fb007b1ce8d9e30d33aa0c33123f178323bda67ac482c9db" gracePeriod=30
Jul 20 17:14:34.134922 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.134888    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:34.358466 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.358403    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:14:34.459728 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.459698    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-maas-api-tls\") pod \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\" (UID: \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\") "
Jul 20 17:14:34.459900 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.459765    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-openshift-service-ca\") pod \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\" (UID: \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\") "
Jul 20 17:14:34.459900 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.459846    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhgbs\" (UniqueName: \"kubernetes.io/projected/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-kube-api-access-bhgbs\") pod \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\" (UID: \"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43\") "
Jul 20 17:14:34.460193 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.460165    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e66d3516-f1eb-4a2d-9e1e-4ccb816faa43" (UID: "e66d3516-f1eb-4a2d-9e1e-4ccb816faa43"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:14:34.461999 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.461975    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e66d3516-f1eb-4a2d-9e1e-4ccb816faa43" (UID: "e66d3516-f1eb-4a2d-9e1e-4ccb816faa43"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:14:34.461999 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.461990    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-kube-api-access-bhgbs" (OuterVolumeSpecName: "kube-api-access-bhgbs") pod "e66d3516-f1eb-4a2d-9e1e-4ccb816faa43" (UID: "e66d3516-f1eb-4a2d-9e1e-4ccb816faa43"). InnerVolumeSpecName "kube-api-access-bhgbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:14:34.560649 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.560610    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-openshift-service-ca\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:14:34.560649 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.560640    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhgbs\" (UniqueName: \"kubernetes.io/projected/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-kube-api-access-bhgbs\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:14:34.560649 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.560651    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43-maas-api-tls\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:14:34.996034 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.995986    2568 generic.go:356] "Generic (PLEG): container finished" podID="e66d3516-f1eb-4a2d-9e1e-4ccb816faa43" containerID="158575e99fb5d640fb007b1ce8d9e30d33aa0c33123f178323bda67ac482c9db" exitCode=0
Jul 20 17:14:34.996034 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.996037    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"
Jul 20 17:14:34.996302 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.996088    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n" event={"ID":"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43","Type":"ContainerDied","Data":"158575e99fb5d640fb007b1ce8d9e30d33aa0c33123f178323bda67ac482c9db"}
Jul 20 17:14:34.996302 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.996144    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n" event={"ID":"e66d3516-f1eb-4a2d-9e1e-4ccb816faa43","Type":"ContainerDied","Data":"a801112d17879b6e9a731656decbab3aee7ebacec8b8644c1733879183aeeedd"}
Jul 20 17:14:34.996302 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:34.996167    2568 scope.go:117] "RemoveContainer" containerID="158575e99fb5d640fb007b1ce8d9e30d33aa0c33123f178323bda67ac482c9db"
Jul 20 17:14:35.005176 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:35.005157    2568 scope.go:117] "RemoveContainer" containerID="158575e99fb5d640fb007b1ce8d9e30d33aa0c33123f178323bda67ac482c9db"
Jul 20 17:14:35.005484 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:14:35.005460    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"158575e99fb5d640fb007b1ce8d9e30d33aa0c33123f178323bda67ac482c9db\": container with ID starting with 158575e99fb5d640fb007b1ce8d9e30d33aa0c33123f178323bda67ac482c9db not found: ID does not exist" containerID="158575e99fb5d640fb007b1ce8d9e30d33aa0c33123f178323bda67ac482c9db"
Jul 20 17:14:35.005539 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:35.005492    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"158575e99fb5d640fb007b1ce8d9e30d33aa0c33123f178323bda67ac482c9db"} err="failed to get container status \"158575e99fb5d640fb007b1ce8d9e30d33aa0c33123f178323bda67ac482c9db\": rpc error: code = NotFound desc = could not find container \"158575e99fb5d640fb007b1ce8d9e30d33aa0c33123f178323bda67ac482c9db\": container with ID starting with 158575e99fb5d640fb007b1ce8d9e30d33aa0c33123f178323bda67ac482c9db not found: ID does not exist"
Jul 20 17:14:35.014478 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:35.014456    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"]
Jul 20 17:14:35.018668 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:35.018643    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9002fce-65ff785b44-t2n7n"]
Jul 20 17:14:36.896207 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:36.896176    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e66d3516-f1eb-4a2d-9e1e-4ccb816faa43" path="/var/lib/kubelet/pods/e66d3516-f1eb-4a2d-9e1e-4ccb816faa43/volumes"
Jul 20 17:14:38.601655 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:38.601619    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:39.809589 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:39.809556    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:40.040362 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:40.040331    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:40.270481 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:40.270418    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:40.613387 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:40.613303    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:40.728223 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:40.728187    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:42.851161 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:42.851132    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:42.947456 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:42.947410    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"]
Jul 20 17:14:42.947990 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:42.947971    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e66d3516-f1eb-4a2d-9e1e-4ccb816faa43" containerName="maas-api"
Jul 20 17:14:42.947990 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:42.947992    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e66d3516-f1eb-4a2d-9e1e-4ccb816faa43" containerName="maas-api"
Jul 20 17:14:42.948180 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:42.948089    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e66d3516-f1eb-4a2d-9e1e-4ccb816faa43" containerName="maas-api"
Jul 20 17:14:42.950260 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:42.950221    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:42.953304 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:42.953285    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:14:42.956040 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:42.956021    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-9185322f-serving-cert\""
Jul 20 17:14:42.962490 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:42.962469    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"]
Jul 20 17:14:43.019430 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.019398    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:43.038583 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.038556    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/341491c8-a189-484c-8d1c-be31b14c456f-openshift-service-ca\") pod \"maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2\" (UID: \"341491c8-a189-484c-8d1c-be31b14c456f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:14:43.038777 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.038598    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/341491c8-a189-484c-8d1c-be31b14c456f-maas-api-tls\") pod \"maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2\" (UID: \"341491c8-a189-484c-8d1c-be31b14c456f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:14:43.038777 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.038732    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2cjd\" (UniqueName: \"kubernetes.io/projected/341491c8-a189-484c-8d1c-be31b14c456f-kube-api-access-n2cjd\") pod \"maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2\" (UID: \"341491c8-a189-484c-8d1c-be31b14c456f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:14:43.139830 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.139797    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2cjd\" (UniqueName: \"kubernetes.io/projected/341491c8-a189-484c-8d1c-be31b14c456f-kube-api-access-n2cjd\") pod \"maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2\" (UID: \"341491c8-a189-484c-8d1c-be31b14c456f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:14:43.140002 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.139882    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/341491c8-a189-484c-8d1c-be31b14c456f-openshift-service-ca\") pod \"maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2\" (UID: \"341491c8-a189-484c-8d1c-be31b14c456f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:14:43.140002 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.139926    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/341491c8-a189-484c-8d1c-be31b14c456f-maas-api-tls\") pod \"maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2\" (UID: \"341491c8-a189-484c-8d1c-be31b14c456f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:14:43.140095 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:14:43.140069    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-serving-cert: secret "maas-api-e2e-derive-9185322f-serving-cert" not found
Jul 20 17:14:43.140147 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:14:43.140136    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/341491c8-a189-484c-8d1c-be31b14c456f-maas-api-tls podName:341491c8-a189-484c-8d1c-be31b14c456f nodeName:}" failed. No retries permitted until 2026-07-20 17:14:43.640116257 +0000 UTC m=+1763.187972306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/341491c8-a189-484c-8d1c-be31b14c456f-maas-api-tls") pod "maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2" (UID: "341491c8-a189-484c-8d1c-be31b14c456f") : secret "maas-api-e2e-derive-9185322f-serving-cert" not found
Jul 20 17:14:43.140567 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.140536    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/341491c8-a189-484c-8d1c-be31b14c456f-openshift-service-ca\") pod \"maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2\" (UID: \"341491c8-a189-484c-8d1c-be31b14c456f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:14:43.149194 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.149168    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2cjd\" (UniqueName: \"kubernetes.io/projected/341491c8-a189-484c-8d1c-be31b14c456f-kube-api-access-n2cjd\") pod \"maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2\" (UID: \"341491c8-a189-484c-8d1c-be31b14c456f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:14:43.406005 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.405903    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:43.572055 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.572023    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:43.646439 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.646403    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/341491c8-a189-484c-8d1c-be31b14c456f-maas-api-tls\") pod \"maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2\" (UID: \"341491c8-a189-484c-8d1c-be31b14c456f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:14:43.648994 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.648971    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/341491c8-a189-484c-8d1c-be31b14c456f-maas-api-tls\") pod \"maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2\" (UID: \"341491c8-a189-484c-8d1c-be31b14c456f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:14:43.717835 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.717736    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:43.865439 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:43.865392    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:14:44.016330 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:44.016305    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"]
Jul 20 17:14:44.018206 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:14:44.018178    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod341491c8_a189_484c_8d1c_be31b14c456f.slice/crio-61d83e68bca127dbf3079eea12518c0c3b8b64e9d8fe4ca9c26242109aa0b3df WatchSource:0}: Error finding container 61d83e68bca127dbf3079eea12518c0c3b8b64e9d8fe4ca9c26242109aa0b3df: Status 404 returned error can't find the container with id 61d83e68bca127dbf3079eea12518c0c3b8b64e9d8fe4ca9c26242109aa0b3df
Jul 20 17:14:44.042654 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:44.042612    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2" event={"ID":"341491c8-a189-484c-8d1c-be31b14c456f","Type":"ContainerStarted","Data":"61d83e68bca127dbf3079eea12518c0c3b8b64e9d8fe4ca9c26242109aa0b3df"}
Jul 20 17:14:45.049035 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:45.048998    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2" event={"ID":"341491c8-a189-484c-8d1c-be31b14c456f","Type":"ContainerStarted","Data":"2308ec395fc425089e07c82f9efa389d3a0b345d002d5ed84a7070ad5963b3c2"}
Jul 20 17:14:45.049462 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:45.049143    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:14:45.074734 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:45.074686    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2" podStartSLOduration=2.202707998 podStartE2EDuration="3.074669576s" totalImagesPullingTime="871.961578ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:14:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:14:45.073230515 +0000 UTC m=+1764.621086580" watchObservedRunningTime="2026-07-20 17:14:45.074669576 +0000 UTC m=+1764.622525652"
Jul 20 17:14:49.960216 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:49.960186    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:50.060598 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:50.060565    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:51.059180 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:14:51.059152    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:15:04.555524 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:04.555482    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:04.700219 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:04.700184    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:18.299534 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:18.299340    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"]
Jul 20 17:15:18.300441 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:18.300134    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2" podUID="341491c8-a189-484c-8d1c-be31b14c456f" containerName="maas-api" containerID="cri-o://2308ec395fc425089e07c82f9efa389d3a0b345d002d5ed84a7070ad5963b3c2" gracePeriod=30
Jul 20 17:15:18.337924 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:18.337894    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:18.561440 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:18.561379    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:15:18.659265 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:18.659224    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2cjd\" (UniqueName: \"kubernetes.io/projected/341491c8-a189-484c-8d1c-be31b14c456f-kube-api-access-n2cjd\") pod \"341491c8-a189-484c-8d1c-be31b14c456f\" (UID: \"341491c8-a189-484c-8d1c-be31b14c456f\") "
Jul 20 17:15:18.659265 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:18.659270    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/341491c8-a189-484c-8d1c-be31b14c456f-openshift-service-ca\") pod \"341491c8-a189-484c-8d1c-be31b14c456f\" (UID: \"341491c8-a189-484c-8d1c-be31b14c456f\") "
Jul 20 17:15:18.659524 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:18.659296    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/341491c8-a189-484c-8d1c-be31b14c456f-maas-api-tls\") pod \"341491c8-a189-484c-8d1c-be31b14c456f\" (UID: \"341491c8-a189-484c-8d1c-be31b14c456f\") "
Jul 20 17:15:18.659593 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:18.659553    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/341491c8-a189-484c-8d1c-be31b14c456f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "341491c8-a189-484c-8d1c-be31b14c456f" (UID: "341491c8-a189-484c-8d1c-be31b14c456f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:15:18.661499 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:18.661477    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/341491c8-a189-484c-8d1c-be31b14c456f-kube-api-access-n2cjd" (OuterVolumeSpecName: "kube-api-access-n2cjd") pod "341491c8-a189-484c-8d1c-be31b14c456f" (UID: "341491c8-a189-484c-8d1c-be31b14c456f"). InnerVolumeSpecName "kube-api-access-n2cjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:15:18.661589 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:18.661522    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/341491c8-a189-484c-8d1c-be31b14c456f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "341491c8-a189-484c-8d1c-be31b14c456f" (UID: "341491c8-a189-484c-8d1c-be31b14c456f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:15:18.760649 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:18.760622    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2cjd\" (UniqueName: \"kubernetes.io/projected/341491c8-a189-484c-8d1c-be31b14c456f-kube-api-access-n2cjd\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:15:18.760649 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:18.760647    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/341491c8-a189-484c-8d1c-be31b14c456f-openshift-service-ca\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:15:18.760649 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:18.760656    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/341491c8-a189-484c-8d1c-be31b14c456f-maas-api-tls\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:15:19.215078 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:19.214990    2568 generic.go:356] "Generic (PLEG): container finished" podID="341491c8-a189-484c-8d1c-be31b14c456f" containerID="2308ec395fc425089e07c82f9efa389d3a0b345d002d5ed84a7070ad5963b3c2" exitCode=0
Jul 20 17:15:19.215078 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:19.215027    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2" event={"ID":"341491c8-a189-484c-8d1c-be31b14c456f","Type":"ContainerDied","Data":"2308ec395fc425089e07c82f9efa389d3a0b345d002d5ed84a7070ad5963b3c2"}
Jul 20 17:15:19.215078 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:19.215036    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"
Jul 20 17:15:19.215078 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:19.215073    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2" event={"ID":"341491c8-a189-484c-8d1c-be31b14c456f","Type":"ContainerDied","Data":"61d83e68bca127dbf3079eea12518c0c3b8b64e9d8fe4ca9c26242109aa0b3df"}
Jul 20 17:15:19.215438 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:19.215100    2568 scope.go:117] "RemoveContainer" containerID="2308ec395fc425089e07c82f9efa389d3a0b345d002d5ed84a7070ad5963b3c2"
Jul 20 17:15:19.223877 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:19.223859    2568 scope.go:117] "RemoveContainer" containerID="2308ec395fc425089e07c82f9efa389d3a0b345d002d5ed84a7070ad5963b3c2"
Jul 20 17:15:19.224121 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:15:19.224096    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2308ec395fc425089e07c82f9efa389d3a0b345d002d5ed84a7070ad5963b3c2\": container with ID starting with 2308ec395fc425089e07c82f9efa389d3a0b345d002d5ed84a7070ad5963b3c2 not found: ID does not exist" containerID="2308ec395fc425089e07c82f9efa389d3a0b345d002d5ed84a7070ad5963b3c2"
Jul 20 17:15:19.224165 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:19.224128    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2308ec395fc425089e07c82f9efa389d3a0b345d002d5ed84a7070ad5963b3c2"} err="failed to get container status \"2308ec395fc425089e07c82f9efa389d3a0b345d002d5ed84a7070ad5963b3c2\": rpc error: code = NotFound desc = could not find container \"2308ec395fc425089e07c82f9efa389d3a0b345d002d5ed84a7070ad5963b3c2\": container with ID starting with 2308ec395fc425089e07c82f9efa389d3a0b345d002d5ed84a7070ad5963b3c2 not found: ID does not exist"
Jul 20 17:15:19.232256 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:19.232223    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"]
Jul 20 17:15:19.236268 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:19.236230    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9185322f-7bc464b4f9-t6dl2"]
Jul 20 17:15:20.896601 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:20.896566    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="341491c8-a189-484c-8d1c-be31b14c456f" path="/var/lib/kubelet/pods/341491c8-a189-484c-8d1c-be31b14c456f/volumes"
Jul 20 17:15:20.931829 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:20.931796    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:15:20.934358 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:20.934337    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:15:22.842603 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:22.842566    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:23.021097 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:23.021062    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:23.262657 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:23.262632    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:23.804577 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:23.804539    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:23.904990 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:23.904951    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:25.393833 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:25.393795    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:32.263216 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:32.263171    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:32.525366 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:32.525264    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:32.647638 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:32.647603    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:37.318168 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:37.318132    2568 generic.go:356] "Generic (PLEG): container finished" podID="9351b069-61d0-490c-a90b-a8b68fa167dd" containerID="1ae74c0d2ef22947b8f09e3549064c86b5b9bc673b395e6326c9d1ed4482349f" exitCode=1
Jul 20 17:15:37.318706 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:37.318185    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk" event={"ID":"9351b069-61d0-490c-a90b-a8b68fa167dd","Type":"ContainerDied","Data":"1ae74c0d2ef22947b8f09e3549064c86b5b9bc673b395e6326c9d1ed4482349f"}
Jul 20 17:15:37.318706 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:37.318598    2568 scope.go:117] "RemoveContainer" containerID="1ae74c0d2ef22947b8f09e3549064c86b5b9bc673b395e6326c9d1ed4482349f"
Jul 20 17:15:37.322502 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:37.322482    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 20 17:15:37.773566 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:37.773470    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m" podUID="954a9eb2-1593-4471-abf5-f30e49776c4c" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.32:8081/readyz\": dial tcp 10.133.0.32:8081: connect: connection refused"
Jul 20 17:15:38.165184 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:38.165147    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:38.325215 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:38.325183    2568 generic.go:356] "Generic (PLEG): container finished" podID="9a98944f-4afb-4716-978a-58b04ff33c50" containerID="c2a1c522809c89fcc693ac0e4259fd8b83c56b16ca18137fbea008f2d2ebfd88" exitCode=1
Jul 20 17:15:38.325764 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:38.325271    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf" event={"ID":"9a98944f-4afb-4716-978a-58b04ff33c50","Type":"ContainerDied","Data":"c2a1c522809c89fcc693ac0e4259fd8b83c56b16ca18137fbea008f2d2ebfd88"}
Jul 20 17:15:38.325764 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:38.325596    2568 scope.go:117] "RemoveContainer" containerID="c2a1c522809c89fcc693ac0e4259fd8b83c56b16ca18137fbea008f2d2ebfd88"
Jul 20 17:15:38.326992 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:38.326971    2568 generic.go:356] "Generic (PLEG): container finished" podID="954a9eb2-1593-4471-abf5-f30e49776c4c" containerID="26d22076be752d24418cd2c6b17bbfd65f1a20890aa28326284f133a44c44ade" exitCode=1
Jul 20 17:15:38.327103 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:38.326993    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m" event={"ID":"954a9eb2-1593-4471-abf5-f30e49776c4c","Type":"ContainerDied","Data":"26d22076be752d24418cd2c6b17bbfd65f1a20890aa28326284f133a44c44ade"}
Jul 20 17:15:38.327301 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:38.327283    2568 scope.go:117] "RemoveContainer" containerID="26d22076be752d24418cd2c6b17bbfd65f1a20890aa28326284f133a44c44ade"
Jul 20 17:15:38.329257 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:38.329215    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk" event={"ID":"9351b069-61d0-490c-a90b-a8b68fa167dd","Type":"ContainerStarted","Data":"e85c42d97fe42de171d277e04490629fcf455b7be8efbc0fc82509cebae0dfd6"}
Jul 20 17:15:38.329732 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:38.329713    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"
Jul 20 17:15:38.369081 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:38.368962    2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 17:15:39.335952 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:39.335859    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf" event={"ID":"9a98944f-4afb-4716-978a-58b04ff33c50","Type":"ContainerStarted","Data":"2b2f8a267450d2813338a6a146f2c57dd8a71cbe218be26278e15bc557ba882f"}
Jul 20 17:15:39.336395 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:39.335957    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 17:15:39.337747 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:39.337724    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m" event={"ID":"954a9eb2-1593-4471-abf5-f30e49776c4c","Type":"ContainerStarted","Data":"180d015c59570f8aa376d3529e9792e03d4e600cb7c975a314192d35da1e5545"}
Jul 20 17:15:39.338027 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:39.338012    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"
Jul 20 17:15:43.357691 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:43.357646    2568 prober.go:120] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-8.ec2.internal" podUID="e1dddf4097058c612737a8bf9968fc66" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 20 17:15:48.385844 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:48.385807    2568 generic.go:356] "Generic (PLEG): container finished" podID="9351b069-61d0-490c-a90b-a8b68fa167dd" containerID="e85c42d97fe42de171d277e04490629fcf455b7be8efbc0fc82509cebae0dfd6" exitCode=1
Jul 20 17:15:48.386273 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:48.385875    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk" event={"ID":"9351b069-61d0-490c-a90b-a8b68fa167dd","Type":"ContainerDied","Data":"e85c42d97fe42de171d277e04490629fcf455b7be8efbc0fc82509cebae0dfd6"}
Jul 20 17:15:48.386273 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:48.385933    2568 scope.go:117] "RemoveContainer" containerID="1ae74c0d2ef22947b8f09e3549064c86b5b9bc673b395e6326c9d1ed4482349f"
Jul 20 17:15:48.386273 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:48.386195    2568 scope.go:117] "RemoveContainer" containerID="e85c42d97fe42de171d277e04490629fcf455b7be8efbc0fc82509cebae0dfd6"
Jul 20 17:15:48.386451 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:15:48.386413    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=dns-operator-controller-manager-864dd78c4c-hfghk_kuadrant-system(9351b069-61d0-490c-a90b-a8b68fa167dd)\"" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk" podUID="9351b069-61d0-490c-a90b-a8b68fa167dd"
Jul 20 17:15:49.392742 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:49.392706    2568 generic.go:356] "Generic (PLEG): container finished" podID="9a98944f-4afb-4716-978a-58b04ff33c50" containerID="2b2f8a267450d2813338a6a146f2c57dd8a71cbe218be26278e15bc557ba882f" exitCode=1
Jul 20 17:15:49.393166 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:49.392786    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf" event={"ID":"9a98944f-4afb-4716-978a-58b04ff33c50","Type":"ContainerDied","Data":"2b2f8a267450d2813338a6a146f2c57dd8a71cbe218be26278e15bc557ba882f"}
Jul 20 17:15:49.393166 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:49.392849    2568 scope.go:117] "RemoveContainer" containerID="c2a1c522809c89fcc693ac0e4259fd8b83c56b16ca18137fbea008f2d2ebfd88"
Jul 20 17:15:49.393313 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:49.393212    2568 scope.go:117] "RemoveContainer" containerID="2b2f8a267450d2813338a6a146f2c57dd8a71cbe218be26278e15bc557ba882f"
Jul 20 17:15:49.393500 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:15:49.393470    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-7998c67dc4-7l9vf_opendatahub(9a98944f-4afb-4716-978a-58b04ff33c50)\"" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf" podUID="9a98944f-4afb-4716-978a-58b04ff33c50"
Jul 20 17:15:49.394862 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:49.394801    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m" event={"ID":"954a9eb2-1593-4471-abf5-f30e49776c4c","Type":"ContainerDied","Data":"180d015c59570f8aa376d3529e9792e03d4e600cb7c975a314192d35da1e5545"}
Jul 20 17:15:49.400268 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:49.394875    2568 generic.go:356] "Generic (PLEG): container finished" podID="954a9eb2-1593-4471-abf5-f30e49776c4c" containerID="180d015c59570f8aa376d3529e9792e03d4e600cb7c975a314192d35da1e5545" exitCode=1
Jul 20 17:15:49.400268 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:49.395616    2568 scope.go:117] "RemoveContainer" containerID="180d015c59570f8aa376d3529e9792e03d4e600cb7c975a314192d35da1e5545"
Jul 20 17:15:49.400268 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:15:49.395953    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=limitador-operator-controller-manager-859d7b78bd-f2n7m_kuadrant-system(954a9eb2-1593-4471-abf5-f30e49776c4c)\"" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m" podUID="954a9eb2-1593-4471-abf5-f30e49776c4c"
Jul 20 17:15:49.409335 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:49.409317    2568 scope.go:117] "RemoveContainer" containerID="26d22076be752d24418cd2c6b17bbfd65f1a20890aa28326284f133a44c44ade"
Jul 20 17:15:54.085717 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:54.085689    2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"
Jul 20 17:15:54.086116 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:54.085990    2568 scope.go:117] "RemoveContainer" containerID="180d015c59570f8aa376d3529e9792e03d4e600cb7c975a314192d35da1e5545"
Jul 20 17:15:54.086185 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:15:54.086166    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=limitador-operator-controller-manager-859d7b78bd-f2n7m_kuadrant-system(954a9eb2-1593-4471-abf5-f30e49776c4c)\"" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m" podUID="954a9eb2-1593-4471-abf5-f30e49776c4c"
Jul 20 17:15:56.586231 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:56.586194    2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"
Jul 20 17:15:56.586707 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:56.586541    2568 scope.go:117] "RemoveContainer" containerID="e85c42d97fe42de171d277e04490629fcf455b7be8efbc0fc82509cebae0dfd6"
Jul 20 17:15:56.586767 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:15:56.586731    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=dns-operator-controller-manager-864dd78c4c-hfghk_kuadrant-system(9351b069-61d0-490c-a90b-a8b68fa167dd)\"" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk" podUID="9351b069-61d0-490c-a90b-a8b68fa167dd"
Jul 20 17:15:58.369196 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:58.369161    2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 17:15:58.369603 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:15:58.369512    2568 scope.go:117] "RemoveContainer" containerID="2b2f8a267450d2813338a6a146f2c57dd8a71cbe218be26278e15bc557ba882f"
Jul 20 17:15:58.369723 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:15:58.369702    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-7998c67dc4-7l9vf_opendatahub(9a98944f-4afb-4716-978a-58b04ff33c50)\"" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf" podUID="9a98944f-4afb-4716-978a-58b04ff33c50"
Jul 20 17:16:04.125014 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:04.124963    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:07.890684 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:07.890645    2568 scope.go:117] "RemoveContainer" containerID="180d015c59570f8aa376d3529e9792e03d4e600cb7c975a314192d35da1e5545"
Jul 20 17:16:07.891059 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:07.890738    2568 scope.go:117] "RemoveContainer" containerID="e85c42d97fe42de171d277e04490629fcf455b7be8efbc0fc82509cebae0dfd6"
Jul 20 17:16:08.494525 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:08.494488    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m" event={"ID":"954a9eb2-1593-4471-abf5-f30e49776c4c","Type":"ContainerStarted","Data":"8c1b91c36cb951198b07a73bc8f9f0b640034320db64bcbde8d953171f2ed66c"}
Jul 20 17:16:08.494789 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:08.494757    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"
Jul 20 17:16:08.496346 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:08.496324    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk" event={"ID":"9351b069-61d0-490c-a90b-a8b68fa167dd","Type":"ContainerStarted","Data":"dbfa0228287d21555774554232aca25ebb3451dc21cb4672f9632a7e7e3b32c6"}
Jul 20 17:16:08.496515 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:08.496501    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"
Jul 20 17:16:09.890593 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:09.890564    2568 scope.go:117] "RemoveContainer" containerID="2b2f8a267450d2813338a6a146f2c57dd8a71cbe218be26278e15bc557ba882f"
Jul 20 17:16:10.428407 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:10.428368    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:10.548272 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:10.548220    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:10.686162 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:10.686135    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:11.512432 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:11.512401    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf" event={"ID":"9a98944f-4afb-4716-978a-58b04ff33c50","Type":"ContainerStarted","Data":"b92e97d0153c296cc05e5819050fb815ac958d4a19b9c9bc34c949e7832894f6"}
Jul 20 17:16:11.512910 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:11.512612    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 17:16:19.503048 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:19.503011    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hfghk"
Jul 20 17:16:19.503630 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:19.503614    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f2n7m"
Jul 20 17:16:22.519630 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:22.519594    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-7l9vf"
Jul 20 17:16:26.600860 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:26.600823    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:32.766854 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:32.766818    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:32.878396 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:32.878365    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:32.977720 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:32.977677    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:43.609221 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:43.609136    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:44.323540 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:44.323506    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:50.702732 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:50.702701    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:50.846553 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:50.846516    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:51.009414 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:51.009315    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:51.808901 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:51.808870    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:51.968149 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:51.968118    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:52.060622 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:52.060529    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:53.680708 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:16:53.680675    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:00.876637 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:00.876603    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:01.014335 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:01.014298    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:01.559603 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:01.559562    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:07.901867 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:07.901829    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:07.967920 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:07.967882    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:08.086362 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:08.086323    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:08.408008 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:08.407962    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:08.829953 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:08.829857    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:11.543442 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.543403    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:11.612455 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.612427    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"]
Jul 20 17:17:11.612794 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.612783    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="341491c8-a189-484c-8d1c-be31b14c456f" containerName="maas-api"
Jul 20 17:17:11.612841 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.612796    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="341491c8-a189-484c-8d1c-be31b14c456f" containerName="maas-api"
Jul 20 17:17:11.612877 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.612871    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="341491c8-a189-484c-8d1c-be31b14c456f" containerName="maas-api"
Jul 20 17:17:11.615834 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.615813    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:11.618360 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.618342    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-fd39db2f-serving-cert\""
Jul 20 17:17:11.627797 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.627771    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"]
Jul 20 17:17:11.632005 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.631978    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:11.697015 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.696980    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:11.763718 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.763687    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-openshift-service-ca\") pod \"maas-api-e2e-mt-fd39db2f-8579bdd985-clskd\" (UID: \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:11.763883 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.763731    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d962g\" (UniqueName: \"kubernetes.io/projected/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-kube-api-access-d962g\") pod \"maas-api-e2e-mt-fd39db2f-8579bdd985-clskd\" (UID: \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:11.763883 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.763754    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-maas-api-tls\") pod \"maas-api-e2e-mt-fd39db2f-8579bdd985-clskd\" (UID: \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:11.798014 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.797938    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:11.864579 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.864549    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-openshift-service-ca\") pod \"maas-api-e2e-mt-fd39db2f-8579bdd985-clskd\" (UID: \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:11.864730 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.864586    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d962g\" (UniqueName: \"kubernetes.io/projected/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-kube-api-access-d962g\") pod \"maas-api-e2e-mt-fd39db2f-8579bdd985-clskd\" (UID: \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:11.864730 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.864610    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-maas-api-tls\") pod \"maas-api-e2e-mt-fd39db2f-8579bdd985-clskd\" (UID: \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:11.865195 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.865174    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-openshift-service-ca\") pod \"maas-api-e2e-mt-fd39db2f-8579bdd985-clskd\" (UID: \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:11.867086 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.867064    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-maas-api-tls\") pod \"maas-api-e2e-mt-fd39db2f-8579bdd985-clskd\" (UID: \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:11.874532 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.874504    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d962g\" (UniqueName: \"kubernetes.io/projected/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-kube-api-access-d962g\") pod \"maas-api-e2e-mt-fd39db2f-8579bdd985-clskd\" (UID: \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:11.927085 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:11.927039    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:12.072452 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:12.072426    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"]
Jul 20 17:17:12.073805 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:17:12.073783    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf20fa755_d1e5_40e7_b8d6_ab41c1a05deb.slice/crio-d453741e7021ca74da1ff2dace3d7fba1c491b9675e32bd4b07706d6f20ed357 WatchSource:0}: Error finding container d453741e7021ca74da1ff2dace3d7fba1c491b9675e32bd4b07706d6f20ed357: Status 404 returned error can't find the container with id d453741e7021ca74da1ff2dace3d7fba1c491b9675e32bd4b07706d6f20ed357
Jul 20 17:17:12.279690 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:12.279656    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:12.448116 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:12.448082    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:12.823526 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:12.823433    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd" event={"ID":"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb","Type":"ContainerStarted","Data":"8e965702923e2130cb0ad0cb9de4174ad445feca600b29d84cad2c596c2e9858"}
Jul 20 17:17:12.823526 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:12.823504    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd" event={"ID":"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb","Type":"ContainerStarted","Data":"d453741e7021ca74da1ff2dace3d7fba1c491b9675e32bd4b07706d6f20ed357"}
Jul 20 17:17:12.823959 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:12.823540    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:12.845046 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:12.844994    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd" podStartSLOduration=1.285929911 podStartE2EDuration="1.844977331s" totalImagesPullingTime="559.04742ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:17:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:17:12.842765696 +0000 UTC m=+1912.390621762" watchObservedRunningTime="2026-07-20 17:17:12.844977331 +0000 UTC m=+1912.392833397"
Jul 20 17:17:15.815466 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:15.815432    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:16.501190 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:16.501157    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:16.587669 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:16.587626    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:16.951362 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:16.951297    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:18.833872 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:18.833841    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:21.150057 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:21.150026    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:22.772167 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:22.772135    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:22.883198 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:22.883162    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:33.855101 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:33.846706    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"]
Jul 20 17:17:33.855101 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:33.846942    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd" podUID="f20fa755-d1e5-40e7-b8d6-ab41c1a05deb" containerName="maas-api" containerID="cri-o://8e965702923e2130cb0ad0cb9de4174ad445feca600b29d84cad2c596c2e9858" gracePeriod=30
Jul 20 17:17:33.928423 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:33.928385    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:34.100470 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.100446    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:34.158508 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.158473    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d962g\" (UniqueName: \"kubernetes.io/projected/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-kube-api-access-d962g\") pod \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\" (UID: \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\") "
Jul 20 17:17:34.158697 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.158527    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-maas-api-tls\") pod \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\" (UID: \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\") "
Jul 20 17:17:34.158697 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.158575    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-openshift-service-ca\") pod \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\" (UID: \"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb\") "
Jul 20 17:17:34.158977 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.158951    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f20fa755-d1e5-40e7-b8d6-ab41c1a05deb" (UID: "f20fa755-d1e5-40e7-b8d6-ab41c1a05deb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:17:34.160831 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.160807    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-kube-api-access-d962g" (OuterVolumeSpecName: "kube-api-access-d962g") pod "f20fa755-d1e5-40e7-b8d6-ab41c1a05deb" (UID: "f20fa755-d1e5-40e7-b8d6-ab41c1a05deb"). InnerVolumeSpecName "kube-api-access-d962g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:17:34.160918 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.160899    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f20fa755-d1e5-40e7-b8d6-ab41c1a05deb" (UID: "f20fa755-d1e5-40e7-b8d6-ab41c1a05deb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:17:34.259168 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.259136    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-openshift-service-ca\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:17:34.259168 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.259162    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d962g\" (UniqueName: \"kubernetes.io/projected/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-kube-api-access-d962g\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:17:34.259168 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.259172    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb-maas-api-tls\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:17:34.928631 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.928596    2568 generic.go:356] "Generic (PLEG): container finished" podID="f20fa755-d1e5-40e7-b8d6-ab41c1a05deb" containerID="8e965702923e2130cb0ad0cb9de4174ad445feca600b29d84cad2c596c2e9858" exitCode=0
Jul 20 17:17:34.929045 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.928640    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"
Jul 20 17:17:34.929045 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.928688    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd" event={"ID":"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb","Type":"ContainerDied","Data":"8e965702923e2130cb0ad0cb9de4174ad445feca600b29d84cad2c596c2e9858"}
Jul 20 17:17:34.929045 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.928755    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd" event={"ID":"f20fa755-d1e5-40e7-b8d6-ab41c1a05deb","Type":"ContainerDied","Data":"d453741e7021ca74da1ff2dace3d7fba1c491b9675e32bd4b07706d6f20ed357"}
Jul 20 17:17:34.929045 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.928789    2568 scope.go:117] "RemoveContainer" containerID="8e965702923e2130cb0ad0cb9de4174ad445feca600b29d84cad2c596c2e9858"
Jul 20 17:17:34.937369 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.937352    2568 scope.go:117] "RemoveContainer" containerID="8e965702923e2130cb0ad0cb9de4174ad445feca600b29d84cad2c596c2e9858"
Jul 20 17:17:34.937629 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:17:34.937612    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e965702923e2130cb0ad0cb9de4174ad445feca600b29d84cad2c596c2e9858\": container with ID starting with 8e965702923e2130cb0ad0cb9de4174ad445feca600b29d84cad2c596c2e9858 not found: ID does not exist" containerID="8e965702923e2130cb0ad0cb9de4174ad445feca600b29d84cad2c596c2e9858"
Jul 20 17:17:34.937689 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.937643    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e965702923e2130cb0ad0cb9de4174ad445feca600b29d84cad2c596c2e9858"} err="failed to get container status \"8e965702923e2130cb0ad0cb9de4174ad445feca600b29d84cad2c596c2e9858\": rpc error: code = NotFound desc = could not find container \"8e965702923e2130cb0ad0cb9de4174ad445feca600b29d84cad2c596c2e9858\": container with ID starting with 8e965702923e2130cb0ad0cb9de4174ad445feca600b29d84cad2c596c2e9858 not found: ID does not exist"
Jul 20 17:17:34.954968 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.954944    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"]
Jul 20 17:17:34.959531 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:34.959509    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fd39db2f-8579bdd985-clskd"]
Jul 20 17:17:36.896039 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:36.896006    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f20fa755-d1e5-40e7-b8d6-ab41c1a05deb" path="/var/lib/kubelet/pods/f20fa755-d1e5-40e7-b8d6-ab41c1a05deb/volumes"
Jul 20 17:17:38.441735 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:38.441700    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:38.845539 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:38.845443    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:44.665918 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:44.665884    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:44.726892 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:44.726865    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:45.025135 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:45.025053    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:45.262854 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:45.262815    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:45.594519 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:45.594481    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:45.695909 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:45.695873    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:51.496080 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:51.496043    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:51.559622 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:51.559590    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:51.645108 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:51.645073    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:51.749039 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:51.748951    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:51.845887 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:51.845860    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:51.947831 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:51.947802    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:57.671427 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:57.671384    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:57.773038 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:17:57.773006    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:04.456626 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:04.456590    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:06.346446 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:06.346412    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:07.458920 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.458884    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:07.555222 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.555183    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:07.636755 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.636717    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"]
Jul 20 17:18:07.637085 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.637072    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f20fa755-d1e5-40e7-b8d6-ab41c1a05deb" containerName="maas-api"
Jul 20 17:18:07.637155 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.637087    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f20fa755-d1e5-40e7-b8d6-ab41c1a05deb" containerName="maas-api"
Jul 20 17:18:07.637209 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.637154    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f20fa755-d1e5-40e7-b8d6-ab41c1a05deb" containerName="maas-api"
Jul 20 17:18:07.640221 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.640202    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:07.643896 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.643869    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-fe047e55-serving-cert\""
Jul 20 17:18:07.653618 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.653596    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"]
Jul 20 17:18:07.666440 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.666399    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6x8h\" (UniqueName: \"kubernetes.io/projected/1342911b-8964-4983-8d15-f8b53d7c00de-kube-api-access-w6x8h\") pod \"maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6\" (UID: \"1342911b-8964-4983-8d15-f8b53d7c00de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:07.666647 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.666624    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1342911b-8964-4983-8d15-f8b53d7c00de-maas-api-tls\") pod \"maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6\" (UID: \"1342911b-8964-4983-8d15-f8b53d7c00de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:07.666759 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.666688    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1342911b-8964-4983-8d15-f8b53d7c00de-openshift-service-ca\") pod \"maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6\" (UID: \"1342911b-8964-4983-8d15-f8b53d7c00de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:07.679843 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.679819    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:07.767191 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.767088    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1342911b-8964-4983-8d15-f8b53d7c00de-openshift-service-ca\") pod \"maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6\" (UID: \"1342911b-8964-4983-8d15-f8b53d7c00de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:07.767191 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.767159    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6x8h\" (UniqueName: \"kubernetes.io/projected/1342911b-8964-4983-8d15-f8b53d7c00de-kube-api-access-w6x8h\") pod \"maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6\" (UID: \"1342911b-8964-4983-8d15-f8b53d7c00de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:07.767446 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.767205    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1342911b-8964-4983-8d15-f8b53d7c00de-maas-api-tls\") pod \"maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6\" (UID: \"1342911b-8964-4983-8d15-f8b53d7c00de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:07.767446 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:18:07.767332    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-serving-cert: secret "maas-api-e2e-mt-fe047e55-serving-cert" not found
Jul 20 17:18:07.767446 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:18:07.767416    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1342911b-8964-4983-8d15-f8b53d7c00de-maas-api-tls podName:1342911b-8964-4983-8d15-f8b53d7c00de nodeName:}" failed. No retries permitted until 2026-07-20 17:18:08.267393799 +0000 UTC m=+1967.815249857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1342911b-8964-4983-8d15-f8b53d7c00de-maas-api-tls") pod "maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6" (UID: "1342911b-8964-4983-8d15-f8b53d7c00de") : secret "maas-api-e2e-mt-fe047e55-serving-cert" not found
Jul 20 17:18:07.767855 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.767833    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1342911b-8964-4983-8d15-f8b53d7c00de-openshift-service-ca\") pod \"maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6\" (UID: \"1342911b-8964-4983-8d15-f8b53d7c00de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:07.781146 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:07.781118    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6x8h\" (UniqueName: \"kubernetes.io/projected/1342911b-8964-4983-8d15-f8b53d7c00de-kube-api-access-w6x8h\") pod \"maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6\" (UID: \"1342911b-8964-4983-8d15-f8b53d7c00de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:08.155599 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:08.155570    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:08.257560 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:08.257528    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:08.271771 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:08.271747    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1342911b-8964-4983-8d15-f8b53d7c00de-maas-api-tls\") pod \"maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6\" (UID: \"1342911b-8964-4983-8d15-f8b53d7c00de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:08.274228 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:08.274203    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1342911b-8964-4983-8d15-f8b53d7c00de-maas-api-tls\") pod \"maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6\" (UID: \"1342911b-8964-4983-8d15-f8b53d7c00de\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:08.357184 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:08.357145    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:08.554734 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:08.554644    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:08.692998 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:08.692972    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"]
Jul 20 17:18:08.695204 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:18:08.695173    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1342911b_8964_4983_8d15_f8b53d7c00de.slice/crio-bc52c5b91e151e00fbbd47fac70d9d7f853d7e7cad1b166a5ab8b719ee069617 WatchSource:0}: Error finding container bc52c5b91e151e00fbbd47fac70d9d7f853d7e7cad1b166a5ab8b719ee069617: Status 404 returned error can't find the container with id bc52c5b91e151e00fbbd47fac70d9d7f853d7e7cad1b166a5ab8b719ee069617
Jul 20 17:18:08.696455 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:08.696439    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 17:18:09.094218 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:09.094192    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6" event={"ID":"1342911b-8964-4983-8d15-f8b53d7c00de","Type":"ContainerStarted","Data":"bc52c5b91e151e00fbbd47fac70d9d7f853d7e7cad1b166a5ab8b719ee069617"}
Jul 20 17:18:10.100580 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:10.100544    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6" event={"ID":"1342911b-8964-4983-8d15-f8b53d7c00de","Type":"ContainerStarted","Data":"8a6bdc58689107f3f8a75274efb07be83dcbc3f99277eb3d5f54fed5b1c68914"}
Jul 20 17:18:10.101004 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:10.100607    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:10.120203 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:10.120152    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6" podStartSLOduration=2.768928083 podStartE2EDuration="3.120138213s" totalImagesPullingTime="351.21013ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:18:10.117636103 +0000 UTC m=+1969.665492171" watchObservedRunningTime="2026-07-20 17:18:10.120138213 +0000 UTC m=+1969.667994279"
Jul 20 17:18:11.965213 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:11.965135    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:16.111720 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:16.111691    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:18:18.899325 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.899290    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"]
Jul 20 17:18:18.902663 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.902641    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:18.913201 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.913162    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fe047e55\"/\"default-dockercfg-wqc58\""
Jul 20 17:18:18.913369 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.913162    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fe047e55\"/\"kube-root-ca.crt\""
Jul 20 17:18:18.913369 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.913167    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fe047e55\"/\"openshift-service-ca.crt\""
Jul 20 17:18:18.913496 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.913165    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fe047e55\"/\"e2e-lifecycle-model-fe047e55-kserve-self-signed-certs\""
Jul 20 17:18:18.915732 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.915707    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"]
Jul 20 17:18:18.936550 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.936519    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:18.964769 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.964717    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-model-cache\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:18.964900 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.964830    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-home\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:18.964900 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.964861    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-dshm\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:18.964988 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.964932    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r8gc\" (UniqueName: \"kubernetes.io/projected/ea57ce70-a9df-4ea2-a442-2e00a872783e-kube-api-access-5r8gc\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:18.964988 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.964963    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-tmp-dir\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:18.965055 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:18.965007    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea57ce70-a9df-4ea2-a442-2e00a872783e-tls-certs\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.066141 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.066098    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea57ce70-a9df-4ea2-a442-2e00a872783e-tls-certs\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.066369 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.066183    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-model-cache\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.066369 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.066233    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-home\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.066369 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.066284    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-dshm\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.066369 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.066323    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r8gc\" (UniqueName: \"kubernetes.io/projected/ea57ce70-a9df-4ea2-a442-2e00a872783e-kube-api-access-5r8gc\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.066369 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.066348    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-tmp-dir\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.066674 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.066651    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-model-cache\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.066740 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.066656    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-home\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.066789 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.066734    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-tmp-dir\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.068543 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.068522    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-dshm\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.068802 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.068784    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea57ce70-a9df-4ea2-a442-2e00a872783e-tls-certs\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.075341 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.075319    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r8gc\" (UniqueName: \"kubernetes.io/projected/ea57ce70-a9df-4ea2-a442-2e00a872783e-kube-api-access-5r8gc\") pod \"e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") " pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.221296 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.221170    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:19.364291 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:19.364265    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"]
Jul 20 17:18:19.366059 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:18:19.366034    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea57ce70_a9df_4ea2_a442_2e00a872783e.slice/crio-2b3911f46b1329a908bd955bcf15c33c9812f965f9659d2b5b15af53ffd89d86 WatchSource:0}: Error finding container 2b3911f46b1329a908bd955bcf15c33c9812f965f9659d2b5b15af53ffd89d86: Status 404 returned error can't find the container with id 2b3911f46b1329a908bd955bcf15c33c9812f965f9659d2b5b15af53ffd89d86
Jul 20 17:18:20.153947 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:20.153911    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq" event={"ID":"ea57ce70-a9df-4ea2-a442-2e00a872783e","Type":"ContainerStarted","Data":"9268c1b60d2dbb76035d5e712de98156da4eb3e17c96eba333087289e44a0f9d"}
Jul 20 17:18:20.154361 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:20.153967    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq" event={"ID":"ea57ce70-a9df-4ea2-a442-2e00a872783e","Type":"ContainerStarted","Data":"2b3911f46b1329a908bd955bcf15c33c9812f965f9659d2b5b15af53ffd89d86"}
Jul 20 17:18:20.175216 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:20.175165    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq" podStartSLOduration=2.175148921 podStartE2EDuration="2.175148921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:18:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:18:20.174447831 +0000 UTC m=+1979.722303896" watchObservedRunningTime="2026-07-20 17:18:20.175148921 +0000 UTC m=+1979.723004986"
Jul 20 17:18:29.222030 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:29.221997    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:29.222030 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:29.222037    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:29.234357 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:29.234333    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:18:30.219525 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:18:30.219496    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:19:01.246110 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.246075    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"]
Jul 20 17:19:01.246699 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.246319    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6" podUID="1342911b-8964-4983-8d15-f8b53d7c00de" containerName="maas-api" containerID="cri-o://8a6bdc58689107f3f8a75274efb07be83dcbc3f99277eb3d5f54fed5b1c68914" gracePeriod=30
Jul 20 17:19:01.330609 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.330580    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:01.361079 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.361047    2568 generic.go:356] "Generic (PLEG): container finished" podID="1342911b-8964-4983-8d15-f8b53d7c00de" containerID="8a6bdc58689107f3f8a75274efb07be83dcbc3f99277eb3d5f54fed5b1c68914" exitCode=0
Jul 20 17:19:01.361260 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.361118    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6" event={"ID":"1342911b-8964-4983-8d15-f8b53d7c00de","Type":"ContainerDied","Data":"8a6bdc58689107f3f8a75274efb07be83dcbc3f99277eb3d5f54fed5b1c68914"}
Jul 20 17:19:01.502426 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.502361    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:19:01.531161 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.531130    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6x8h\" (UniqueName: \"kubernetes.io/projected/1342911b-8964-4983-8d15-f8b53d7c00de-kube-api-access-w6x8h\") pod \"1342911b-8964-4983-8d15-f8b53d7c00de\" (UID: \"1342911b-8964-4983-8d15-f8b53d7c00de\") "
Jul 20 17:19:01.531370 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.531184    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1342911b-8964-4983-8d15-f8b53d7c00de-openshift-service-ca\") pod \"1342911b-8964-4983-8d15-f8b53d7c00de\" (UID: \"1342911b-8964-4983-8d15-f8b53d7c00de\") "
Jul 20 17:19:01.531370 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.531222    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1342911b-8964-4983-8d15-f8b53d7c00de-maas-api-tls\") pod \"1342911b-8964-4983-8d15-f8b53d7c00de\" (UID: \"1342911b-8964-4983-8d15-f8b53d7c00de\") "
Jul 20 17:19:01.531561 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.531533    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1342911b-8964-4983-8d15-f8b53d7c00de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1342911b-8964-4983-8d15-f8b53d7c00de" (UID: "1342911b-8964-4983-8d15-f8b53d7c00de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:19:01.533361 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.533334    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1342911b-8964-4983-8d15-f8b53d7c00de-kube-api-access-w6x8h" (OuterVolumeSpecName: "kube-api-access-w6x8h") pod "1342911b-8964-4983-8d15-f8b53d7c00de" (UID: "1342911b-8964-4983-8d15-f8b53d7c00de"). InnerVolumeSpecName "kube-api-access-w6x8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:19:01.533495 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.533378    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1342911b-8964-4983-8d15-f8b53d7c00de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1342911b-8964-4983-8d15-f8b53d7c00de" (UID: "1342911b-8964-4983-8d15-f8b53d7c00de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:19:01.631839 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.631804    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1342911b-8964-4983-8d15-f8b53d7c00de-maas-api-tls\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:19:01.631839 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.631837    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6x8h\" (UniqueName: \"kubernetes.io/projected/1342911b-8964-4983-8d15-f8b53d7c00de-kube-api-access-w6x8h\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:19:01.631839 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:01.631846    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1342911b-8964-4983-8d15-f8b53d7c00de-openshift-service-ca\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:19:02.367329 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:02.367299    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"
Jul 20 17:19:02.367329 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:02.367311    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6" event={"ID":"1342911b-8964-4983-8d15-f8b53d7c00de","Type":"ContainerDied","Data":"bc52c5b91e151e00fbbd47fac70d9d7f853d7e7cad1b166a5ab8b719ee069617"}
Jul 20 17:19:02.367836 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:02.367373    2568 scope.go:117] "RemoveContainer" containerID="8a6bdc58689107f3f8a75274efb07be83dcbc3f99277eb3d5f54fed5b1c68914"
Jul 20 17:19:02.390639 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:02.390613    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"]
Jul 20 17:19:02.394807 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:02.394787    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fe047e55-7565d96cfd-ntcx6"]
Jul 20 17:19:02.896496 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:02.896465    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1342911b-8964-4983-8d15-f8b53d7c00de" path="/var/lib/kubelet/pods/1342911b-8964-4983-8d15-f8b53d7c00de/volumes"
Jul 20 17:19:03.659161 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:03.659124    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:05.818031 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:05.817995    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:08.085997 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:08.085964    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:13.062377 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:13.062340    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"]
Jul 20 17:19:13.155830 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:13.155799    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:22.788956 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:22.788916    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:28.082931 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.082885    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq" podUID="ea57ce70-a9df-4ea2-a442-2e00a872783e" containerName="main" containerID="cri-o://9268c1b60d2dbb76035d5e712de98156da4eb3e17c96eba333087289e44a0f9d" gracePeriod=45
Jul 20 17:19:28.340187 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.340126    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:19:28.469181 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.469141    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-dshm\") pod \"ea57ce70-a9df-4ea2-a442-2e00a872783e\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") "
Jul 20 17:19:28.469409 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.469218    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5r8gc\" (UniqueName: \"kubernetes.io/projected/ea57ce70-a9df-4ea2-a442-2e00a872783e-kube-api-access-5r8gc\") pod \"ea57ce70-a9df-4ea2-a442-2e00a872783e\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") "
Jul 20 17:19:28.469409 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.469286    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-tmp-dir\") pod \"ea57ce70-a9df-4ea2-a442-2e00a872783e\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") "
Jul 20 17:19:28.469409 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.469393    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-home\") pod \"ea57ce70-a9df-4ea2-a442-2e00a872783e\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") "
Jul 20 17:19:28.469579 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.469432    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-model-cache\") pod \"ea57ce70-a9df-4ea2-a442-2e00a872783e\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") "
Jul 20 17:19:28.469579 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.469469    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea57ce70-a9df-4ea2-a442-2e00a872783e-tls-certs\") pod \"ea57ce70-a9df-4ea2-a442-2e00a872783e\" (UID: \"ea57ce70-a9df-4ea2-a442-2e00a872783e\") "
Jul 20 17:19:28.469579 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.469517    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ea57ce70-a9df-4ea2-a442-2e00a872783e" (UID: "ea57ce70-a9df-4ea2-a442-2e00a872783e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:19:28.469693 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.469656    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-home" (OuterVolumeSpecName: "home") pod "ea57ce70-a9df-4ea2-a442-2e00a872783e" (UID: "ea57ce70-a9df-4ea2-a442-2e00a872783e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:19:28.469693 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.469673    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-model-cache" (OuterVolumeSpecName: "model-cache") pod "ea57ce70-a9df-4ea2-a442-2e00a872783e" (UID: "ea57ce70-a9df-4ea2-a442-2e00a872783e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:19:28.469786 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.469770    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-tmp-dir\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:19:28.469830 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.469793    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-home\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:19:28.469830 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.469806    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-model-cache\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:19:28.471601 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.471581    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea57ce70-a9df-4ea2-a442-2e00a872783e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ea57ce70-a9df-4ea2-a442-2e00a872783e" (UID: "ea57ce70-a9df-4ea2-a442-2e00a872783e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:19:28.471687 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.471618    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-dshm" (OuterVolumeSpecName: "dshm") pod "ea57ce70-a9df-4ea2-a442-2e00a872783e" (UID: "ea57ce70-a9df-4ea2-a442-2e00a872783e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:19:28.471737 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.471684    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea57ce70-a9df-4ea2-a442-2e00a872783e-kube-api-access-5r8gc" (OuterVolumeSpecName: "kube-api-access-5r8gc") pod "ea57ce70-a9df-4ea2-a442-2e00a872783e" (UID: "ea57ce70-a9df-4ea2-a442-2e00a872783e"). InnerVolumeSpecName "kube-api-access-5r8gc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:19:28.501365 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.501338    2568 generic.go:356] "Generic (PLEG): container finished" podID="ea57ce70-a9df-4ea2-a442-2e00a872783e" containerID="9268c1b60d2dbb76035d5e712de98156da4eb3e17c96eba333087289e44a0f9d" exitCode=0
Jul 20 17:19:28.501488 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.501366    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq" event={"ID":"ea57ce70-a9df-4ea2-a442-2e00a872783e","Type":"ContainerDied","Data":"9268c1b60d2dbb76035d5e712de98156da4eb3e17c96eba333087289e44a0f9d"}
Jul 20 17:19:28.501488 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.501395    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"
Jul 20 17:19:28.501488 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.501407    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq" event={"ID":"ea57ce70-a9df-4ea2-a442-2e00a872783e","Type":"ContainerDied","Data":"2b3911f46b1329a908bd955bcf15c33c9812f965f9659d2b5b15af53ffd89d86"}
Jul 20 17:19:28.501488 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.501426    2568 scope.go:117] "RemoveContainer" containerID="9268c1b60d2dbb76035d5e712de98156da4eb3e17c96eba333087289e44a0f9d"
Jul 20 17:19:28.510768 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.510752    2568 scope.go:117] "RemoveContainer" containerID="9268c1b60d2dbb76035d5e712de98156da4eb3e17c96eba333087289e44a0f9d"
Jul 20 17:19:28.511035 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:19:28.511018    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9268c1b60d2dbb76035d5e712de98156da4eb3e17c96eba333087289e44a0f9d\": container with ID starting with 9268c1b60d2dbb76035d5e712de98156da4eb3e17c96eba333087289e44a0f9d not found: ID does not exist" containerID="9268c1b60d2dbb76035d5e712de98156da4eb3e17c96eba333087289e44a0f9d"
Jul 20 17:19:28.511094 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.511047    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9268c1b60d2dbb76035d5e712de98156da4eb3e17c96eba333087289e44a0f9d"} err="failed to get container status \"9268c1b60d2dbb76035d5e712de98156da4eb3e17c96eba333087289e44a0f9d\": rpc error: code = NotFound desc = could not find container \"9268c1b60d2dbb76035d5e712de98156da4eb3e17c96eba333087289e44a0f9d\": container with ID starting with 9268c1b60d2dbb76035d5e712de98156da4eb3e17c96eba333087289e44a0f9d not found: ID does not exist"
Jul 20 17:19:28.523978 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.523957    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"]
Jul 20 17:19:28.528690 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.528672    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-fe047e55/e2e-lifecycle-model-fe047e55-kserve-8ddd55fc5-4cxsq"]
Jul 20 17:19:28.570748 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.570720    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea57ce70-a9df-4ea2-a442-2e00a872783e-tls-certs\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:19:28.570748 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.570743    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea57ce70-a9df-4ea2-a442-2e00a872783e-dshm\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:19:28.570891 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.570756    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r8gc\" (UniqueName: \"kubernetes.io/projected/ea57ce70-a9df-4ea2-a442-2e00a872783e-kube-api-access-5r8gc\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:19:28.897672 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:28.897643    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea57ce70-a9df-4ea2-a442-2e00a872783e" path="/var/lib/kubelet/pods/ea57ce70-a9df-4ea2-a442-2e00a872783e/volumes"
Jul 20 17:19:50.029611 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:50.029576    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:50.401006 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:50.400975    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:50.871484 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:50.871453    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:50.972008 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:50.971977    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:56.166388 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:56.166349    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:56.234392 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:56.234359    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:56.466834 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:56.466749    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:56.692972 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:56.692936    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:57.737517 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:57.737472    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:57.837094 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:57.837060    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:57.935814 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:57.935784    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:58.660750 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:58.660716    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:58.755970 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:58.755940    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:19:58.855733 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:19:58.855664    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:00.235768 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:00.235724    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:05.987378 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:05.987344    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:06.058116 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:06.058079    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:06.664922 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:06.664892    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:12.506385 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:12.506352    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:12.605111 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:12.605080    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:20.960623 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:20.960590    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:20:20.964347 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:20.964326    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:20:23.038909 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:23.038874    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:23.381945 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:23.381917    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:42.469612 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:42.469573    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:44.368580 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:44.368542    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:50.533978 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:50.533940    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:50.633916 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:50.633882    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:50.906814 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:50.906779    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:51.246861 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:51.246777    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:51.496129 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:51.496092    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:54.310630 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.310591    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:54.452146 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.452115    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:54.452146 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.452136    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"]
Jul 20 17:20:54.452617 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.452602    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1342911b-8964-4983-8d15-f8b53d7c00de" containerName="maas-api"
Jul 20 17:20:54.452617 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.452619    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1342911b-8964-4983-8d15-f8b53d7c00de" containerName="maas-api"
Jul 20 17:20:54.452730 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.452627    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea57ce70-a9df-4ea2-a442-2e00a872783e" containerName="main"
Jul 20 17:20:54.452730 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.452634    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea57ce70-a9df-4ea2-a442-2e00a872783e" containerName="main"
Jul 20 17:20:54.452837 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.452768    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1342911b-8964-4983-8d15-f8b53d7c00de" containerName="maas-api"
Jul 20 17:20:54.452837 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.452784    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea57ce70-a9df-4ea2-a442-2e00a872783e" containerName="main"
Jul 20 17:20:54.455822 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.455807    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:20:54.458651 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.458631    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-1d7b9a-serving-cert\""
Jul 20 17:20:54.470055 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.470005    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"]
Jul 20 17:20:54.530554 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.530526    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61e5bb08-42c2-4e6d-b59f-b68e082981ac-maas-api-tls\") pod \"maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24\" (UID: \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:20:54.530721 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.530563    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61e5bb08-42c2-4e6d-b59f-b68e082981ac-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24\" (UID: \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:20:54.530721 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.530594    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6xx2\" (UniqueName: \"kubernetes.io/projected/61e5bb08-42c2-4e6d-b59f-b68e082981ac-kube-api-access-p6xx2\") pod \"maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24\" (UID: \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:20:54.550300 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.550269    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:54.631261 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.631211    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6xx2\" (UniqueName: \"kubernetes.io/projected/61e5bb08-42c2-4e6d-b59f-b68e082981ac-kube-api-access-p6xx2\") pod \"maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24\" (UID: \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:20:54.631469 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.631321    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61e5bb08-42c2-4e6d-b59f-b68e082981ac-maas-api-tls\") pod \"maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24\" (UID: \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:20:54.631469 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.631369    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61e5bb08-42c2-4e6d-b59f-b68e082981ac-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24\" (UID: \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:20:54.631601 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:20:54.631495    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-serving-cert: secret "maas-api-e2e-inf-a-1d7b9a-serving-cert" not found
Jul 20 17:20:54.631601 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:20:54.631581    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61e5bb08-42c2-4e6d-b59f-b68e082981ac-maas-api-tls podName:61e5bb08-42c2-4e6d-b59f-b68e082981ac nodeName:}" failed. No retries permitted until 2026-07-20 17:20:55.131558641 +0000 UTC m=+2134.679414691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/61e5bb08-42c2-4e6d-b59f-b68e082981ac-maas-api-tls") pod "maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24" (UID: "61e5bb08-42c2-4e6d-b59f-b68e082981ac") : secret "maas-api-e2e-inf-a-1d7b9a-serving-cert" not found
Jul 20 17:20:54.632047 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.632028    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61e5bb08-42c2-4e6d-b59f-b68e082981ac-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24\" (UID: \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:20:54.642510 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.642476    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6xx2\" (UniqueName: \"kubernetes.io/projected/61e5bb08-42c2-4e6d-b59f-b68e082981ac-kube-api-access-p6xx2\") pod \"maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24\" (UID: \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:20:54.804178 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:54.804143    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:55.135835 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:55.135809    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61e5bb08-42c2-4e6d-b59f-b68e082981ac-maas-api-tls\") pod \"maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24\" (UID: \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:20:55.138212 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:55.138190    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61e5bb08-42c2-4e6d-b59f-b68e082981ac-maas-api-tls\") pod \"maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24\" (UID: \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:20:55.146492 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:55.146468    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:55.239822 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:55.239790    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:55.365623 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:55.365587    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:20:55.711309 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:55.711286    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"]
Jul 20 17:20:55.713375 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:20:55.713335    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61e5bb08_42c2_4e6d_b59f_b68e082981ac.slice/crio-efc476279290ac6bcb7bf5d0997d2dad13cb448ed0be5bd90c12da4eb53a58a3 WatchSource:0}: Error finding container efc476279290ac6bcb7bf5d0997d2dad13cb448ed0be5bd90c12da4eb53a58a3: Status 404 returned error can't find the container with id efc476279290ac6bcb7bf5d0997d2dad13cb448ed0be5bd90c12da4eb53a58a3
Jul 20 17:20:55.938959 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:55.938927    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24" event={"ID":"61e5bb08-42c2-4e6d-b59f-b68e082981ac","Type":"ContainerStarted","Data":"efc476279290ac6bcb7bf5d0997d2dad13cb448ed0be5bd90c12da4eb53a58a3"}
Jul 20 17:20:56.946304 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:56.946234    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24" event={"ID":"61e5bb08-42c2-4e6d-b59f-b68e082981ac","Type":"ContainerStarted","Data":"b90d7ca9ba3f9d4c31354269ec250435c3e6b1c973a94e476b86457b1647b638"}
Jul 20 17:20:56.946684 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:56.946333    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:20:56.989156 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:56.989101    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24" podStartSLOduration=2.426514428 podStartE2EDuration="2.989085571s" totalImagesPullingTime="562.571143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:20:56.98832785 +0000 UTC m=+2136.536183917" watchObservedRunningTime="2026-07-20 17:20:56.989085571 +0000 UTC m=+2136.536941637"
Jul 20 17:20:58.778847 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:58.778811    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:59.159670 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:59.159641    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:59.499978 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:59.499887    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:20:59.814003 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:20:59.813921    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:02.451363 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.451328    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:02.546869 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.546833    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:02.614093 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.614051    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"]
Jul 20 17:21:02.618025 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.618001    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:02.620834 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.620808    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-0062e2-serving-cert\""
Jul 20 17:21:02.631866 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.631843    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"]
Jul 20 17:21:02.670340 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.670298    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:02.712626 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.712528    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8662f3-7bc8-4048-8888-ec83de4f8828-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh\" (UID: \"9e8662f3-7bc8-4048-8888-ec83de4f8828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:02.712626 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.712580    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e8662f3-7bc8-4048-8888-ec83de4f8828-maas-api-tls\") pod \"maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh\" (UID: \"9e8662f3-7bc8-4048-8888-ec83de4f8828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:02.712875 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.712698    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7987p\" (UniqueName: \"kubernetes.io/projected/9e8662f3-7bc8-4048-8888-ec83de4f8828-kube-api-access-7987p\") pod \"maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh\" (UID: \"9e8662f3-7bc8-4048-8888-ec83de4f8828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:02.813206 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.813169    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8662f3-7bc8-4048-8888-ec83de4f8828-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh\" (UID: \"9e8662f3-7bc8-4048-8888-ec83de4f8828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:02.813434 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.813216    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e8662f3-7bc8-4048-8888-ec83de4f8828-maas-api-tls\") pod \"maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh\" (UID: \"9e8662f3-7bc8-4048-8888-ec83de4f8828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:02.813434 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.813285    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7987p\" (UniqueName: \"kubernetes.io/projected/9e8662f3-7bc8-4048-8888-ec83de4f8828-kube-api-access-7987p\") pod \"maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh\" (UID: \"9e8662f3-7bc8-4048-8888-ec83de4f8828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:02.813434 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:21:02.813407    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-serving-cert: secret "maas-api-e2e-inf-b-0062e2-serving-cert" not found
Jul 20 17:21:02.813608 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:21:02.813474    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e8662f3-7bc8-4048-8888-ec83de4f8828-maas-api-tls podName:9e8662f3-7bc8-4048-8888-ec83de4f8828 nodeName:}" failed. No retries permitted until 2026-07-20 17:21:03.313451753 +0000 UTC m=+2142.861307799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9e8662f3-7bc8-4048-8888-ec83de4f8828-maas-api-tls") pod "maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh" (UID: "9e8662f3-7bc8-4048-8888-ec83de4f8828") : secret "maas-api-e2e-inf-b-0062e2-serving-cert" not found
Jul 20 17:21:02.813949 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.813929    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8662f3-7bc8-4048-8888-ec83de4f8828-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh\" (UID: \"9e8662f3-7bc8-4048-8888-ec83de4f8828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:02.830385 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.830356    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7987p\" (UniqueName: \"kubernetes.io/projected/9e8662f3-7bc8-4048-8888-ec83de4f8828-kube-api-access-7987p\") pod \"maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh\" (UID: \"9e8662f3-7bc8-4048-8888-ec83de4f8828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:02.933055 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.933013    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:02.956131 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:02.956104    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:21:03.275206 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:03.275174    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:03.319213 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:03.319181    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e8662f3-7bc8-4048-8888-ec83de4f8828-maas-api-tls\") pod \"maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh\" (UID: \"9e8662f3-7bc8-4048-8888-ec83de4f8828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:03.321916 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:03.321891    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e8662f3-7bc8-4048-8888-ec83de4f8828-maas-api-tls\") pod \"maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh\" (UID: \"9e8662f3-7bc8-4048-8888-ec83de4f8828\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:03.377300 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:03.377269    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:03.530653 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:03.530568    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:03.678138 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:03.678114    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"]
Jul 20 17:21:03.680562 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:21:03.680529    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8662f3_7bc8_4048_8888_ec83de4f8828.slice/crio-ae4239fabc4e8981b6affd2de56c8475cd090206c31226e049e3c62f7b8342b6 WatchSource:0}: Error finding container ae4239fabc4e8981b6affd2de56c8475cd090206c31226e049e3c62f7b8342b6: Status 404 returned error can't find the container with id ae4239fabc4e8981b6affd2de56c8475cd090206c31226e049e3c62f7b8342b6
Jul 20 17:21:03.979484 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:03.979453    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh" event={"ID":"9e8662f3-7bc8-4048-8888-ec83de4f8828","Type":"ContainerStarted","Data":"ae4239fabc4e8981b6affd2de56c8475cd090206c31226e049e3c62f7b8342b6"}
Jul 20 17:21:04.985575 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:04.985531    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh" event={"ID":"9e8662f3-7bc8-4048-8888-ec83de4f8828","Type":"ContainerStarted","Data":"a75962664f191c97cb0c5ede3697c18f262d1ad429b9d2c7f8a53f0c7922d0a0"}
Jul 20 17:21:04.986000 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:04.985640    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:05.009459 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:05.009406    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh" podStartSLOduration=2.631121205 podStartE2EDuration="3.009388422s" totalImagesPullingTime="378.267217ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:21:05.007172115 +0000 UTC m=+2144.555028180" watchObservedRunningTime="2026-07-20 17:21:05.009388422 +0000 UTC m=+2144.557244487"
Jul 20 17:21:06.974518 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:06.974481    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:11.002193 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:11.002163    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:21:16.066073 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:16.065986    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:16.166986 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:16.166940    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:19.899269 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.899217    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"]
Jul 20 17:21:19.906275 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.905592    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:19.910229 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.910203    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1d7b9a\"/\"default-dockercfg-m7wf4\""
Jul 20 17:21:19.910473 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.910213    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1d7b9a\"/\"openshift-service-ca.crt\""
Jul 20 17:21:19.910547 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.910230    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1d7b9a\"/\"test-model-1d7b9a-kserve-self-signed-certs\""
Jul 20 17:21:19.910547 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.910323    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1d7b9a\"/\"kube-root-ca.crt\""
Jul 20 17:21:19.912333 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.912307    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"]
Jul 20 17:21:19.917439 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.917409    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:19.961636 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.961595    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-model-cache\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:19.961636 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.961639    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-tmp-dir\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:19.961846 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.961690    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghmzs\" (UniqueName: \"kubernetes.io/projected/72bcd131-8216-47b0-a862-7370488ff899-kube-api-access-ghmzs\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:19.961846 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.961712    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72bcd131-8216-47b0-a862-7370488ff899-tls-certs\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:19.961846 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.961761    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-home\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:19.961846 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:19.961785    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-dshm\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.062601 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.062570    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-model-cache\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.062601 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.062606    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-tmp-dir\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.062863 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.062660    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghmzs\" (UniqueName: \"kubernetes.io/projected/72bcd131-8216-47b0-a862-7370488ff899-kube-api-access-ghmzs\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.062863 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.062686    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72bcd131-8216-47b0-a862-7370488ff899-tls-certs\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.062863 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.062707    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-home\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.062863 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.062726    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-dshm\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.063072 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.063049    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-model-cache\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.063121 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.063079    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-home\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.063197 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.063176    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-tmp-dir\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.065199 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.065178    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-dshm\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.065404 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.065388    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72bcd131-8216-47b0-a862-7370488ff899-tls-certs\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.073121 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.073083    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghmzs\" (UniqueName: \"kubernetes.io/projected/72bcd131-8216-47b0-a862-7370488ff899-kube-api-access-ghmzs\") pod \"test-model-1d7b9a-kserve-569df8f6cc-6bkcq\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") " pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.216909 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.216821    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:20.358682 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:20.358658    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"]
Jul 20 17:21:20.360924 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:21:20.360892    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72bcd131_8216_47b0_a862_7370488ff899.slice/crio-f24651f335e39004edd9c661300c23c18dc7a267d2220c854e1788a095f7e1d5 WatchSource:0}: Error finding container f24651f335e39004edd9c661300c23c18dc7a267d2220c854e1788a095f7e1d5: Status 404 returned error can't find the container with id f24651f335e39004edd9c661300c23c18dc7a267d2220c854e1788a095f7e1d5
Jul 20 17:21:21.070635 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:21.070598    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq" event={"ID":"72bcd131-8216-47b0-a862-7370488ff899","Type":"ContainerStarted","Data":"7c390013e661cf487c3adaa212431d35feb25c458f6548512ad9e17ff82bfc72"}
Jul 20 17:21:21.071036 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:21.070642    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq" event={"ID":"72bcd131-8216-47b0-a862-7370488ff899","Type":"ContainerStarted","Data":"f24651f335e39004edd9c661300c23c18dc7a267d2220c854e1788a095f7e1d5"}
Jul 20 17:21:21.100769 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:21.100718    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq" podStartSLOduration=2.100700448 podStartE2EDuration="2.100700448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:21:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:21:21.099120893 +0000 UTC m=+2160.646976958" watchObservedRunningTime="2026-07-20 17:21:21.100700448 +0000 UTC m=+2160.648556513"
Jul 20 17:21:24.056646 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:24.056616    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:24.157489 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:24.157460    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:30.217965 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:30.217924    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:30.217965 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:30.217973    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:30.230596 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:30.230570    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:31.131934 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:31.131896    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:21:35.444059 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:35.444022    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:35.508414 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:35.508383    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:58.283512 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.283479    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"]
Jul 20 17:21:58.287325 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.287303    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.291102 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.291074    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0062e2\"/\"default-dockercfg-k4zp8\""
Jul 20 17:21:58.291255 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.291081    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0062e2\"/\"test-model-0062e2-kserve-self-signed-certs\""
Jul 20 17:21:58.291255 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.291086    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0062e2\"/\"openshift-service-ca.crt\""
Jul 20 17:21:58.291255 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.291145    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0062e2\"/\"kube-root-ca.crt\""
Jul 20 17:21:58.298084 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.298065    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"]
Jul 20 17:21:58.307631 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.307607    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:21:58.397714 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.397680    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-model-cache\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.397882 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.397720    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-dshm\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.397882 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.397747    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-tmp-dir\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.397882 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.397806    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-home\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.397882 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.397848    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-tls-certs\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.397882 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.397867    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpjgf\" (UniqueName: \"kubernetes.io/projected/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-kube-api-access-hpjgf\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.498938 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.498900    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-home\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.498938 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.498943    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-tls-certs\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.499184 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.498963    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpjgf\" (UniqueName: \"kubernetes.io/projected/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-kube-api-access-hpjgf\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.499184 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.499027    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-model-cache\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.499184 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.499052    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-dshm\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.499184 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.499088    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-tmp-dir\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.499451 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.499415    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-model-cache\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.499528 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.499498    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-home\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.499528 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.499513    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-tmp-dir\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.501517 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.501494    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-dshm\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.501784 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.501767    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-tls-certs\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.507513 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.507485    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpjgf\" (UniqueName: \"kubernetes.io/projected/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-kube-api-access-hpjgf\") pod \"test-model-0062e2-kserve-687d898459-hqpg7\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") " pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.597491 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.597405    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:21:58.746847 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:58.746825    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"]
Jul 20 17:21:58.749602 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:21:58.749572    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb27e5a2f_b6b9_4b66_bee0_93fc4de3dd02.slice/crio-f462c26721d89d5c4b0c2e57482f96aee90545bc4ff62899057e4feb3ea67373 WatchSource:0}: Error finding container f462c26721d89d5c4b0c2e57482f96aee90545bc4ff62899057e4feb3ea67373: Status 404 returned error can't find the container with id f462c26721d89d5c4b0c2e57482f96aee90545bc4ff62899057e4feb3ea67373
Jul 20 17:21:59.261048 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:59.261014    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7" event={"ID":"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02","Type":"ContainerStarted","Data":"e9e2d4ff3fc7be805c7cb41a8e192749fa7b6247fee6186ad17d7dd90ae00f53"}
Jul 20 17:21:59.261048 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:59.261052    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7" event={"ID":"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02","Type":"ContainerStarted","Data":"f462c26721d89d5c4b0c2e57482f96aee90545bc4ff62899057e4feb3ea67373"}
Jul 20 17:21:59.279707 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:21:59.279663    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7" podStartSLOduration=1.279645976 podStartE2EDuration="1.279645976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:21:59.278379102 +0000 UTC m=+2198.826235167" watchObservedRunningTime="2026-07-20 17:21:59.279645976 +0000 UTC m=+2198.827502041"
Jul 20 17:22:08.598145 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:08.598104    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:22:08.598587 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:08.598163    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:22:08.610339 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:08.610314    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:22:09.325213 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:09.325182    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:22:47.330512 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:47.330433    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"]
Jul 20 17:22:47.356468 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:47.356431    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:22:48.293320 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:48.293287    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"]
Jul 20 17:22:48.307651 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:48.307622    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:22:49.047647 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:49.047609    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:22:49.158173 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:49.158144    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:22:58.912462 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:58.912432    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"]
Jul 20 17:22:58.912944 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:58.912697    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24" podUID="61e5bb08-42c2-4e6d-b59f-b68e082981ac" containerName="maas-api" containerID="cri-o://b90d7ca9ba3f9d4c31354269ec250435c3e6b1c973a94e476b86457b1647b638" gracePeriod=30
Jul 20 17:22:58.937426 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:58.937397    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:22:59.170120 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.170058    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:22:59.337694 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.337657    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6xx2\" (UniqueName: \"kubernetes.io/projected/61e5bb08-42c2-4e6d-b59f-b68e082981ac-kube-api-access-p6xx2\") pod \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\" (UID: \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\") "
Jul 20 17:22:59.337885 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.337755    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61e5bb08-42c2-4e6d-b59f-b68e082981ac-maas-api-tls\") pod \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\" (UID: \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\") "
Jul 20 17:22:59.337885 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.337821    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61e5bb08-42c2-4e6d-b59f-b68e082981ac-openshift-service-ca\") pod \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\" (UID: \"61e5bb08-42c2-4e6d-b59f-b68e082981ac\") "
Jul 20 17:22:59.338184 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.338157    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61e5bb08-42c2-4e6d-b59f-b68e082981ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "61e5bb08-42c2-4e6d-b59f-b68e082981ac" (UID: "61e5bb08-42c2-4e6d-b59f-b68e082981ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:22:59.339904 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.339879    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61e5bb08-42c2-4e6d-b59f-b68e082981ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "61e5bb08-42c2-4e6d-b59f-b68e082981ac" (UID: "61e5bb08-42c2-4e6d-b59f-b68e082981ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:22:59.339986 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.339971    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61e5bb08-42c2-4e6d-b59f-b68e082981ac-kube-api-access-p6xx2" (OuterVolumeSpecName: "kube-api-access-p6xx2") pod "61e5bb08-42c2-4e6d-b59f-b68e082981ac" (UID: "61e5bb08-42c2-4e6d-b59f-b68e082981ac"). InnerVolumeSpecName "kube-api-access-p6xx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:22:59.438943 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.438863    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61e5bb08-42c2-4e6d-b59f-b68e082981ac-openshift-service-ca\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:22:59.438943 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.438890    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6xx2\" (UniqueName: \"kubernetes.io/projected/61e5bb08-42c2-4e6d-b59f-b68e082981ac-kube-api-access-p6xx2\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:22:59.438943 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.438904    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61e5bb08-42c2-4e6d-b59f-b68e082981ac-maas-api-tls\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:22:59.564219 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.564185    2568 generic.go:356] "Generic (PLEG): container finished" podID="61e5bb08-42c2-4e6d-b59f-b68e082981ac" containerID="b90d7ca9ba3f9d4c31354269ec250435c3e6b1c973a94e476b86457b1647b638" exitCode=0
Jul 20 17:22:59.564415 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.564254    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24" event={"ID":"61e5bb08-42c2-4e6d-b59f-b68e082981ac","Type":"ContainerDied","Data":"b90d7ca9ba3f9d4c31354269ec250435c3e6b1c973a94e476b86457b1647b638"}
Jul 20 17:22:59.564415 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.564234    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"
Jul 20 17:22:59.564415 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.564294    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24" event={"ID":"61e5bb08-42c2-4e6d-b59f-b68e082981ac","Type":"ContainerDied","Data":"efc476279290ac6bcb7bf5d0997d2dad13cb448ed0be5bd90c12da4eb53a58a3"}
Jul 20 17:22:59.564415 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.564317    2568 scope.go:117] "RemoveContainer" containerID="b90d7ca9ba3f9d4c31354269ec250435c3e6b1c973a94e476b86457b1647b638"
Jul 20 17:22:59.573852 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.573833    2568 scope.go:117] "RemoveContainer" containerID="b90d7ca9ba3f9d4c31354269ec250435c3e6b1c973a94e476b86457b1647b638"
Jul 20 17:22:59.574129 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:22:59.574108    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b90d7ca9ba3f9d4c31354269ec250435c3e6b1c973a94e476b86457b1647b638\": container with ID starting with b90d7ca9ba3f9d4c31354269ec250435c3e6b1c973a94e476b86457b1647b638 not found: ID does not exist" containerID="b90d7ca9ba3f9d4c31354269ec250435c3e6b1c973a94e476b86457b1647b638"
Jul 20 17:22:59.574208 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.574134    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b90d7ca9ba3f9d4c31354269ec250435c3e6b1c973a94e476b86457b1647b638"} err="failed to get container status \"b90d7ca9ba3f9d4c31354269ec250435c3e6b1c973a94e476b86457b1647b638\": rpc error: code = NotFound desc = could not find container \"b90d7ca9ba3f9d4c31354269ec250435c3e6b1c973a94e476b86457b1647b638\": container with ID starting with b90d7ca9ba3f9d4c31354269ec250435c3e6b1c973a94e476b86457b1647b638 not found: ID does not exist"
Jul 20 17:22:59.586776 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.586747    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"]
Jul 20 17:22:59.594749 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:22:59.594728    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1d7b9a-65cd4fc6f7-cnk24"]
Jul 20 17:23:00.896361 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:00.896328    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61e5bb08-42c2-4e6d-b59f-b68e082981ac" path="/var/lib/kubelet/pods/61e5bb08-42c2-4e6d-b59f-b68e082981ac/volumes"
Jul 20 17:23:02.351814 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.351774    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq" podUID="72bcd131-8216-47b0-a862-7370488ff899" containerName="main" containerID="cri-o://7c390013e661cf487c3adaa212431d35feb25c458f6548512ad9e17ff82bfc72" gracePeriod=45
Jul 20 17:23:02.580497 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.580467    2568 generic.go:356] "Generic (PLEG): container finished" podID="72bcd131-8216-47b0-a862-7370488ff899" containerID="7c390013e661cf487c3adaa212431d35feb25c458f6548512ad9e17ff82bfc72" exitCode=0
Jul 20 17:23:02.580658 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.580541    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq" event={"ID":"72bcd131-8216-47b0-a862-7370488ff899","Type":"ContainerDied","Data":"7c390013e661cf487c3adaa212431d35feb25c458f6548512ad9e17ff82bfc72"}
Jul 20 17:23:02.608327 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.608271    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:23:02.766803 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.766770    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-model-cache\") pod \"72bcd131-8216-47b0-a862-7370488ff899\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") "
Jul 20 17:23:02.766971 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.766834    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72bcd131-8216-47b0-a862-7370488ff899-tls-certs\") pod \"72bcd131-8216-47b0-a862-7370488ff899\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") "
Jul 20 17:23:02.766971 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.766877    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-tmp-dir\") pod \"72bcd131-8216-47b0-a862-7370488ff899\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") "
Jul 20 17:23:02.766971 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.766913    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-dshm\") pod \"72bcd131-8216-47b0-a862-7370488ff899\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") "
Jul 20 17:23:02.766971 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.766959    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-home\") pod \"72bcd131-8216-47b0-a862-7370488ff899\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") "
Jul 20 17:23:02.767201 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.767012    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghmzs\" (UniqueName: \"kubernetes.io/projected/72bcd131-8216-47b0-a862-7370488ff899-kube-api-access-ghmzs\") pod \"72bcd131-8216-47b0-a862-7370488ff899\" (UID: \"72bcd131-8216-47b0-a862-7370488ff899\") "
Jul 20 17:23:02.767201 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.767066    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-model-cache" (OuterVolumeSpecName: "model-cache") pod "72bcd131-8216-47b0-a862-7370488ff899" (UID: "72bcd131-8216-47b0-a862-7370488ff899"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:23:02.767201 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.767140    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "72bcd131-8216-47b0-a862-7370488ff899" (UID: "72bcd131-8216-47b0-a862-7370488ff899"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:23:02.767379 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.767219    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-home" (OuterVolumeSpecName: "home") pod "72bcd131-8216-47b0-a862-7370488ff899" (UID: "72bcd131-8216-47b0-a862-7370488ff899"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:23:02.767379 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.767325    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-tmp-dir\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:02.767379 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.767344    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-home\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:02.767379 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.767357    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-model-cache\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:02.769264 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.769220    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-dshm" (OuterVolumeSpecName: "dshm") pod "72bcd131-8216-47b0-a862-7370488ff899" (UID: "72bcd131-8216-47b0-a862-7370488ff899"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:23:02.769360 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.769291    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72bcd131-8216-47b0-a862-7370488ff899-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "72bcd131-8216-47b0-a862-7370488ff899" (UID: "72bcd131-8216-47b0-a862-7370488ff899"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:23:02.769360 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.769347    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72bcd131-8216-47b0-a862-7370488ff899-kube-api-access-ghmzs" (OuterVolumeSpecName: "kube-api-access-ghmzs") pod "72bcd131-8216-47b0-a862-7370488ff899" (UID: "72bcd131-8216-47b0-a862-7370488ff899"). InnerVolumeSpecName "kube-api-access-ghmzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:23:02.868523 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.868438    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72bcd131-8216-47b0-a862-7370488ff899-dshm\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:02.868523 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.868466    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghmzs\" (UniqueName: \"kubernetes.io/projected/72bcd131-8216-47b0-a862-7370488ff899-kube-api-access-ghmzs\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:02.868523 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:02.868478    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72bcd131-8216-47b0-a862-7370488ff899-tls-certs\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:03.314789 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.314752    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7" podUID="b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02" containerName="main" containerID="cri-o://e9e2d4ff3fc7be805c7cb41a8e192749fa7b6247fee6186ad17d7dd90ae00f53" gracePeriod=45
Jul 20 17:23:03.508572 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.508528    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:03.572414 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.572358    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:23:03.586115 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.586088    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"
Jul 20 17:23:03.586235 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.586138    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq" event={"ID":"72bcd131-8216-47b0-a862-7370488ff899","Type":"ContainerDied","Data":"f24651f335e39004edd9c661300c23c18dc7a267d2220c854e1788a095f7e1d5"}
Jul 20 17:23:03.586235 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.586186    2568 scope.go:117] "RemoveContainer" containerID="7c390013e661cf487c3adaa212431d35feb25c458f6548512ad9e17ff82bfc72"
Jul 20 17:23:03.587421 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.587401    2568 generic.go:356] "Generic (PLEG): container finished" podID="b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02" containerID="e9e2d4ff3fc7be805c7cb41a8e192749fa7b6247fee6186ad17d7dd90ae00f53" exitCode=0
Jul 20 17:23:03.587494 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.587436    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"
Jul 20 17:23:03.587494 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.587437    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7" event={"ID":"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02","Type":"ContainerDied","Data":"e9e2d4ff3fc7be805c7cb41a8e192749fa7b6247fee6186ad17d7dd90ae00f53"}
Jul 20 17:23:03.587494 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.587475    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7" event={"ID":"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02","Type":"ContainerDied","Data":"f462c26721d89d5c4b0c2e57482f96aee90545bc4ff62899057e4feb3ea67373"}
Jul 20 17:23:03.594703 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.594681    2568 scope.go:117] "RemoveContainer" containerID="e9e2d4ff3fc7be805c7cb41a8e192749fa7b6247fee6186ad17d7dd90ae00f53"
Jul 20 17:23:03.604229 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.604212    2568 scope.go:117] "RemoveContainer" containerID="e9e2d4ff3fc7be805c7cb41a8e192749fa7b6247fee6186ad17d7dd90ae00f53"
Jul 20 17:23:03.604537 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:23:03.604518    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9e2d4ff3fc7be805c7cb41a8e192749fa7b6247fee6186ad17d7dd90ae00f53\": container with ID starting with e9e2d4ff3fc7be805c7cb41a8e192749fa7b6247fee6186ad17d7dd90ae00f53 not found: ID does not exist" containerID="e9e2d4ff3fc7be805c7cb41a8e192749fa7b6247fee6186ad17d7dd90ae00f53"
Jul 20 17:23:03.604602 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.604545    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9e2d4ff3fc7be805c7cb41a8e192749fa7b6247fee6186ad17d7dd90ae00f53"} err="failed to get container status \"e9e2d4ff3fc7be805c7cb41a8e192749fa7b6247fee6186ad17d7dd90ae00f53\": rpc error: code = NotFound desc = could not find container \"e9e2d4ff3fc7be805c7cb41a8e192749fa7b6247fee6186ad17d7dd90ae00f53\": container with ID starting with e9e2d4ff3fc7be805c7cb41a8e192749fa7b6247fee6186ad17d7dd90ae00f53 not found: ID does not exist"
Jul 20 17:23:03.612459 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.612438    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"]
Jul 20 17:23:03.617055 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.617030    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-1d7b9a/test-model-1d7b9a-kserve-569df8f6cc-6bkcq"]
Jul 20 17:23:03.674749 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.674714    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-tmp-dir\") pod \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") "
Jul 20 17:23:03.674942 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.674763    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-home\") pod \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") "
Jul 20 17:23:03.674942 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.674799    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-dshm\") pod \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") "
Jul 20 17:23:03.674942 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.674816    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-tls-certs\") pod \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") "
Jul 20 17:23:03.674942 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.674853    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-model-cache\") pod \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") "
Jul 20 17:23:03.674942 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.674874    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpjgf\" (UniqueName: \"kubernetes.io/projected/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-kube-api-access-hpjgf\") pod \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\" (UID: \"b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02\") "
Jul 20 17:23:03.675301 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.675066    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-home" (OuterVolumeSpecName: "home") pod "b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02" (UID: "b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:23:03.675301 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.675075    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02" (UID: "b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:23:03.675301 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.675162    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-model-cache" (OuterVolumeSpecName: "model-cache") pod "b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02" (UID: "b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:23:03.675301 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.675199    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-home\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:03.675301 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.675211    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-tmp-dir\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:03.677312 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.677287    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02" (UID: "b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:23:03.677312 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.677296    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-dshm" (OuterVolumeSpecName: "dshm") pod "b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02" (UID: "b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:23:03.677458 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.677323    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-kube-api-access-hpjgf" (OuterVolumeSpecName: "kube-api-access-hpjgf") pod "b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02" (UID: "b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02"). InnerVolumeSpecName "kube-api-access-hpjgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:23:03.776146 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.776112    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-dshm\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:03.776146 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.776138    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-tls-certs\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:03.776146 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.776148    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-model-cache\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:03.776398 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.776160    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpjgf\" (UniqueName: \"kubernetes.io/projected/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02-kube-api-access-hpjgf\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:03.902049 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.902018    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:03.909753 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.909712    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"]
Jul 20 17:23:03.912062 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:03.912037    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-0062e2/test-model-0062e2-kserve-687d898459-hqpg7"]
Jul 20 17:23:04.896798 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:04.896766    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72bcd131-8216-47b0-a862-7370488ff899" path="/var/lib/kubelet/pods/72bcd131-8216-47b0-a862-7370488ff899/volumes"
Jul 20 17:23:04.897266 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:04.897127    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02" path="/var/lib/kubelet/pods/b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02/volumes"
Jul 20 17:23:10.239651 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:10.239623    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:11.705603 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:11.705564    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:21.307920 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.307883    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"]
Jul 20 17:23:21.309285 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.309224    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh" podUID="9e8662f3-7bc8-4048-8888-ec83de4f8828" containerName="maas-api" containerID="cri-o://a75962664f191c97cb0c5ede3697c18f262d1ad429b9d2c7f8a53f0c7922d0a0" gracePeriod=30
Jul 20 17:23:21.343693 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.343659    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:21.557796 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.557769    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:23:21.629567 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.629540    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e8662f3-7bc8-4048-8888-ec83de4f8828-maas-api-tls\") pod \"9e8662f3-7bc8-4048-8888-ec83de4f8828\" (UID: \"9e8662f3-7bc8-4048-8888-ec83de4f8828\") "
Jul 20 17:23:21.629756 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.629586    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7987p\" (UniqueName: \"kubernetes.io/projected/9e8662f3-7bc8-4048-8888-ec83de4f8828-kube-api-access-7987p\") pod \"9e8662f3-7bc8-4048-8888-ec83de4f8828\" (UID: \"9e8662f3-7bc8-4048-8888-ec83de4f8828\") "
Jul 20 17:23:21.629756 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.629611    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8662f3-7bc8-4048-8888-ec83de4f8828-openshift-service-ca\") pod \"9e8662f3-7bc8-4048-8888-ec83de4f8828\" (UID: \"9e8662f3-7bc8-4048-8888-ec83de4f8828\") "
Jul 20 17:23:21.629971 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.629945    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8662f3-7bc8-4048-8888-ec83de4f8828-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9e8662f3-7bc8-4048-8888-ec83de4f8828" (UID: "9e8662f3-7bc8-4048-8888-ec83de4f8828"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:23:21.631798 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.631776    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e8662f3-7bc8-4048-8888-ec83de4f8828-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9e8662f3-7bc8-4048-8888-ec83de4f8828" (UID: "9e8662f3-7bc8-4048-8888-ec83de4f8828"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:23:21.631885 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.631812    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e8662f3-7bc8-4048-8888-ec83de4f8828-kube-api-access-7987p" (OuterVolumeSpecName: "kube-api-access-7987p") pod "9e8662f3-7bc8-4048-8888-ec83de4f8828" (UID: "9e8662f3-7bc8-4048-8888-ec83de4f8828"). InnerVolumeSpecName "kube-api-access-7987p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:23:21.678459 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.678376    2568 generic.go:356] "Generic (PLEG): container finished" podID="9e8662f3-7bc8-4048-8888-ec83de4f8828" containerID="a75962664f191c97cb0c5ede3697c18f262d1ad429b9d2c7f8a53f0c7922d0a0" exitCode=0
Jul 20 17:23:21.678459 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.678422    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"
Jul 20 17:23:21.678615 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.678466    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh" event={"ID":"9e8662f3-7bc8-4048-8888-ec83de4f8828","Type":"ContainerDied","Data":"a75962664f191c97cb0c5ede3697c18f262d1ad429b9d2c7f8a53f0c7922d0a0"}
Jul 20 17:23:21.678615 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.678511    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh" event={"ID":"9e8662f3-7bc8-4048-8888-ec83de4f8828","Type":"ContainerDied","Data":"ae4239fabc4e8981b6affd2de56c8475cd090206c31226e049e3c62f7b8342b6"}
Jul 20 17:23:21.678615 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.678538    2568 scope.go:117] "RemoveContainer" containerID="a75962664f191c97cb0c5ede3697c18f262d1ad429b9d2c7f8a53f0c7922d0a0"
Jul 20 17:23:21.687898 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.687882    2568 scope.go:117] "RemoveContainer" containerID="a75962664f191c97cb0c5ede3697c18f262d1ad429b9d2c7f8a53f0c7922d0a0"
Jul 20 17:23:21.688141 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:23:21.688121    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a75962664f191c97cb0c5ede3697c18f262d1ad429b9d2c7f8a53f0c7922d0a0\": container with ID starting with a75962664f191c97cb0c5ede3697c18f262d1ad429b9d2c7f8a53f0c7922d0a0 not found: ID does not exist" containerID="a75962664f191c97cb0c5ede3697c18f262d1ad429b9d2c7f8a53f0c7922d0a0"
Jul 20 17:23:21.688274 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.688150    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a75962664f191c97cb0c5ede3697c18f262d1ad429b9d2c7f8a53f0c7922d0a0"} err="failed to get container status \"a75962664f191c97cb0c5ede3697c18f262d1ad429b9d2c7f8a53f0c7922d0a0\": rpc error: code = NotFound desc = could not find container \"a75962664f191c97cb0c5ede3697c18f262d1ad429b9d2c7f8a53f0c7922d0a0\": container with ID starting with a75962664f191c97cb0c5ede3697c18f262d1ad429b9d2c7f8a53f0c7922d0a0 not found: ID does not exist"
Jul 20 17:23:21.702027 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.702006    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"]
Jul 20 17:23:21.707939 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.707912    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0062e2-7b468fc4cf-9vqgh"]
Jul 20 17:23:21.730558 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.730535    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7987p\" (UniqueName: \"kubernetes.io/projected/9e8662f3-7bc8-4048-8888-ec83de4f8828-kube-api-access-7987p\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:21.730558 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.730554    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8662f3-7bc8-4048-8888-ec83de4f8828-openshift-service-ca\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:21.730674 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:21.730564    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e8662f3-7bc8-4048-8888-ec83de4f8828-maas-api-tls\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:23:22.897016 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:22.896983    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e8662f3-7bc8-4048-8888-ec83de4f8828" path="/var/lib/kubelet/pods/9e8662f3-7bc8-4048-8888-ec83de4f8828/volumes"
Jul 20 17:23:25.936712 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:25.936677    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:26.329100 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:26.329002    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:32.717045 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:32.717017    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:32.815738 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:32.815701    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:33.014204 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:33.014112    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:33.213588 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:33.213542    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:33.559282 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:33.559232    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:33.894899 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:33.894864    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:34.288958 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:34.288866    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:36.962641 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:36.962606    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:37.076310 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.076276    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:37.204167 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.204137    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"]
Jul 20 17:23:37.204564 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.204550    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02" containerName="main"
Jul 20 17:23:37.204635 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.204567    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02" containerName="main"
Jul 20 17:23:37.204635 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.204580    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72bcd131-8216-47b0-a862-7370488ff899" containerName="main"
Jul 20 17:23:37.204635 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.204589    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="72bcd131-8216-47b0-a862-7370488ff899" containerName="main"
Jul 20 17:23:37.204635 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.204600    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61e5bb08-42c2-4e6d-b59f-b68e082981ac" containerName="maas-api"
Jul 20 17:23:37.204635 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.204605    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="61e5bb08-42c2-4e6d-b59f-b68e082981ac" containerName="maas-api"
Jul 20 17:23:37.204635 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.204616    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8662f3-7bc8-4048-8888-ec83de4f8828" containerName="maas-api"
Jul 20 17:23:37.204635 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.204621    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8662f3-7bc8-4048-8888-ec83de4f8828" containerName="maas-api"
Jul 20 17:23:37.204991 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.204682    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8662f3-7bc8-4048-8888-ec83de4f8828" containerName="maas-api"
Jul 20 17:23:37.204991 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.204691    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b27e5a2f-b6b9-4b66-bee0-93fc4de3dd02" containerName="main"
Jul 20 17:23:37.204991 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.204699    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="61e5bb08-42c2-4e6d-b59f-b68e082981ac" containerName="maas-api"
Jul 20 17:23:37.204991 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.204705    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="72bcd131-8216-47b0-a862-7370488ff899" containerName="main"
Jul 20 17:23:37.207726 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.207708    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:23:37.210265 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.210218    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-77a9c7-serving-cert\""
Jul 20 17:23:37.220070 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.220046    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"]
Jul 20 17:23:37.239072 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.239030    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:37.265039 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.265009    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-maas-api-tls\") pod \"maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2\" (UID: \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:23:37.265184 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.265042    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-openshift-service-ca\") pod \"maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2\" (UID: \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:23:37.265184 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.265070    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmhcw\" (UniqueName: \"kubernetes.io/projected/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-kube-api-access-pmhcw\") pod \"maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2\" (UID: \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:23:37.366043 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.366012    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-maas-api-tls\") pod \"maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2\" (UID: \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:23:37.366043 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.366048    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-openshift-service-ca\") pod \"maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2\" (UID: \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:23:37.366330 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.366074    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmhcw\" (UniqueName: \"kubernetes.io/projected/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-kube-api-access-pmhcw\") pod \"maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2\" (UID: \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:23:37.366852 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.366782    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-openshift-service-ca\") pod \"maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2\" (UID: \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:23:37.368660 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.368640    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-maas-api-tls\") pod \"maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2\" (UID: \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:23:37.389016 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.388986    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmhcw\" (UniqueName: \"kubernetes.io/projected/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-kube-api-access-pmhcw\") pod \"maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2\" (UID: \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:23:37.436446 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.436414    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:37.517799 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.517713    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:23:37.675321 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.675275    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"]
Jul 20 17:23:37.677454 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:23:37.677427    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc742c3e2_a0d5_4b0a_8ba4_5cf2b52908c9.slice/crio-685b1f2c719cb96de2242e1cdde9cfda1b6edcf8aa413b71713e1f53fdd7ef13 WatchSource:0}: Error finding container 685b1f2c719cb96de2242e1cdde9cfda1b6edcf8aa413b71713e1f53fdd7ef13: Status 404 returned error can't find the container with id 685b1f2c719cb96de2242e1cdde9cfda1b6edcf8aa413b71713e1f53fdd7ef13
Jul 20 17:23:37.679066 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.679051    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 17:23:37.759941 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:37.759909    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2" event={"ID":"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9","Type":"ContainerStarted","Data":"685b1f2c719cb96de2242e1cdde9cfda1b6edcf8aa413b71713e1f53fdd7ef13"}
Jul 20 17:23:38.053313 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:38.053279    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:38.193493 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:38.193460    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:39.772193 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:39.772156    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2" event={"ID":"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9","Type":"ContainerStarted","Data":"0ad8f7dad6e270483aec529844e215cca91fdd9ba009689ccf977344e2f9a2b8"}
Jul 20 17:23:39.772601 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:39.772301    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:23:39.799645 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:39.799594    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2" podStartSLOduration=1.5388575 podStartE2EDuration="2.79957997s" totalImagesPullingTime="1.26072247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:23:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:23:39.797652306 +0000 UTC m=+2299.345508386" watchObservedRunningTime="2026-07-20 17:23:39.79957997 +0000 UTC m=+2299.347436035"
Jul 20 17:23:41.466126 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:41.466092    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:41.944820 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:41.944790    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:42.257185 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:42.257096    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:42.354045 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:42.353990    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:43.095787 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:43.095750    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:45.073060 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.073023    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"]
Jul 20 17:23:45.084344 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.083139    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:23:45.086587 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.086562    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-c63878-serving-cert\""
Jul 20 17:23:45.090363 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.090333    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"]
Jul 20 17:23:45.236821 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.236785    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8l26\" (UniqueName: \"kubernetes.io/projected/ec8ac321-2d7e-4f75-8082-a06152cd6712-kube-api-access-f8l26\") pod \"maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575\" (UID: \"ec8ac321-2d7e-4f75-8082-a06152cd6712\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:23:45.237004 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.236840    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec8ac321-2d7e-4f75-8082-a06152cd6712-openshift-service-ca\") pod \"maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575\" (UID: \"ec8ac321-2d7e-4f75-8082-a06152cd6712\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:23:45.237004 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.236864    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec8ac321-2d7e-4f75-8082-a06152cd6712-maas-api-tls\") pod \"maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575\" (UID: \"ec8ac321-2d7e-4f75-8082-a06152cd6712\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:23:45.306490 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.306452    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:45.337843 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.337760    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec8ac321-2d7e-4f75-8082-a06152cd6712-openshift-service-ca\") pod \"maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575\" (UID: \"ec8ac321-2d7e-4f75-8082-a06152cd6712\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:23:45.337843 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.337802    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec8ac321-2d7e-4f75-8082-a06152cd6712-maas-api-tls\") pod \"maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575\" (UID: \"ec8ac321-2d7e-4f75-8082-a06152cd6712\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:23:45.338022 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.337856    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8l26\" (UniqueName: \"kubernetes.io/projected/ec8ac321-2d7e-4f75-8082-a06152cd6712-kube-api-access-f8l26\") pod \"maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575\" (UID: \"ec8ac321-2d7e-4f75-8082-a06152cd6712\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:23:45.338552 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.338532    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec8ac321-2d7e-4f75-8082-a06152cd6712-openshift-service-ca\") pod \"maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575\" (UID: \"ec8ac321-2d7e-4f75-8082-a06152cd6712\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:23:45.340531 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.340502    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec8ac321-2d7e-4f75-8082-a06152cd6712-maas-api-tls\") pod \"maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575\" (UID: \"ec8ac321-2d7e-4f75-8082-a06152cd6712\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:23:45.348441 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.348413    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8l26\" (UniqueName: \"kubernetes.io/projected/ec8ac321-2d7e-4f75-8082-a06152cd6712-kube-api-access-f8l26\") pod \"maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575\" (UID: \"ec8ac321-2d7e-4f75-8082-a06152cd6712\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:23:45.399919 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.399879    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:23:45.443038 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.442991    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:45.557327 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.557301    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"]
Jul 20 17:23:45.559466 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:23:45.559435    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec8ac321_2d7e_4f75_8082_a06152cd6712.slice/crio-82d597b9f1bdda65a38b66a3039b43fb046c360c1d9eda46accf8100bdc29611 WatchSource:0}: Error finding container 82d597b9f1bdda65a38b66a3039b43fb046c360c1d9eda46accf8100bdc29611: Status 404 returned error can't find the container with id 82d597b9f1bdda65a38b66a3039b43fb046c360c1d9eda46accf8100bdc29611
Jul 20 17:23:45.750288 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.750234    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:45.782358 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.782328    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:23:45.802416 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.802312    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575" event={"ID":"ec8ac321-2d7e-4f75-8082-a06152cd6712","Type":"ContainerStarted","Data":"82d597b9f1bdda65a38b66a3039b43fb046c360c1d9eda46accf8100bdc29611"}
Jul 20 17:23:45.854222 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:45.854189    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:46.808729 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:46.808693    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575" event={"ID":"ec8ac321-2d7e-4f75-8082-a06152cd6712","Type":"ContainerStarted","Data":"ec8664c8488b26df310cc7fad9868e10149787668a122bec0dadc5889e511b10"}
Jul 20 17:23:46.809114 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:46.808758    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:23:46.827421 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:46.827375    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575" podStartSLOduration=1.5014479330000001 podStartE2EDuration="1.82735955s" totalImagesPullingTime="325.911617ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:23:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:23:46.825820915 +0000 UTC m=+2306.373676981" watchObservedRunningTime="2026-07-20 17:23:46.82735955 +0000 UTC m=+2306.375215615"
Jul 20 17:23:49.514892 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:49.514861    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:52.818867 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:52.818832    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:23:58.065695 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:58.065646    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:23:58.167430 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:23:58.167392    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:06.703823 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:06.703793    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:06.803076 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:06.803038    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:12.546476 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.546228    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"]
Jul 20 17:24:12.546930 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.546595    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2" podUID="c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9" containerName="maas-api" containerID="cri-o://0ad8f7dad6e270483aec529844e215cca91fdd9ba009689ccf977344e2f9a2b8" gracePeriod=30
Jul 20 17:24:12.582197 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.582163    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:12.806213 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.806156    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:24:12.868521 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.868488    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-maas-api-tls\") pod \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\" (UID: \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\") "
Jul 20 17:24:12.868521 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.868526    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmhcw\" (UniqueName: \"kubernetes.io/projected/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-kube-api-access-pmhcw\") pod \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\" (UID: \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\") "
Jul 20 17:24:12.868772 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.868580    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-openshift-service-ca\") pod \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\" (UID: \"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9\") "
Jul 20 17:24:12.868969 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.868945    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9" (UID: "c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:24:12.870645 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.870623    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9" (UID: "c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:24:12.870843 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.870821    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-kube-api-access-pmhcw" (OuterVolumeSpecName: "kube-api-access-pmhcw") pod "c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9" (UID: "c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9"). InnerVolumeSpecName "kube-api-access-pmhcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:24:12.935956 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.935925    2568 generic.go:356] "Generic (PLEG): container finished" podID="c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9" containerID="0ad8f7dad6e270483aec529844e215cca91fdd9ba009689ccf977344e2f9a2b8" exitCode=0
Jul 20 17:24:12.936126 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.935972    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"
Jul 20 17:24:12.936126 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.936017    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2" event={"ID":"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9","Type":"ContainerDied","Data":"0ad8f7dad6e270483aec529844e215cca91fdd9ba009689ccf977344e2f9a2b8"}
Jul 20 17:24:12.936126 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.936069    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2" event={"ID":"c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9","Type":"ContainerDied","Data":"685b1f2c719cb96de2242e1cdde9cfda1b6edcf8aa413b71713e1f53fdd7ef13"}
Jul 20 17:24:12.936126 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.936092    2568 scope.go:117] "RemoveContainer" containerID="0ad8f7dad6e270483aec529844e215cca91fdd9ba009689ccf977344e2f9a2b8"
Jul 20 17:24:12.946151 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.946133    2568 scope.go:117] "RemoveContainer" containerID="0ad8f7dad6e270483aec529844e215cca91fdd9ba009689ccf977344e2f9a2b8"
Jul 20 17:24:12.946524 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:24:12.946501    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ad8f7dad6e270483aec529844e215cca91fdd9ba009689ccf977344e2f9a2b8\": container with ID starting with 0ad8f7dad6e270483aec529844e215cca91fdd9ba009689ccf977344e2f9a2b8 not found: ID does not exist" containerID="0ad8f7dad6e270483aec529844e215cca91fdd9ba009689ccf977344e2f9a2b8"
Jul 20 17:24:12.946602 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.946537    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ad8f7dad6e270483aec529844e215cca91fdd9ba009689ccf977344e2f9a2b8"} err="failed to get container status \"0ad8f7dad6e270483aec529844e215cca91fdd9ba009689ccf977344e2f9a2b8\": rpc error: code = NotFound desc = could not find container \"0ad8f7dad6e270483aec529844e215cca91fdd9ba009689ccf977344e2f9a2b8\": container with ID starting with 0ad8f7dad6e270483aec529844e215cca91fdd9ba009689ccf977344e2f9a2b8 not found: ID does not exist"
Jul 20 17:24:12.958422 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.958400    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"]
Jul 20 17:24:12.969535 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.969511    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-77a9c7-85db97576d-fq7s2"]
Jul 20 17:24:12.969927 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.969897    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-openshift-service-ca\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:24:12.969927 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.969924    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-maas-api-tls\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:24:12.970106 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:12.969939    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmhcw\" (UniqueName: \"kubernetes.io/projected/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9-kube-api-access-pmhcw\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:24:14.896327 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:14.896288    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9" path="/var/lib/kubelet/pods/c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9/volumes"
Jul 20 17:24:17.586034 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:17.585996    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:17.971903 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:17.971865    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:24.945362 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:24.945325    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:25.077489 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:25.077452    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:25.193405 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:25.193356    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:25.537883 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:25.537841    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:41.104413 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:41.104369    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"]
Jul 20 17:24:41.105032 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:41.104620    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575" podUID="ec8ac321-2d7e-4f75-8082-a06152cd6712" containerName="maas-api" containerID="cri-o://ec8664c8488b26df310cc7fad9868e10149787668a122bec0dadc5889e511b10" gracePeriod=30
Jul 20 17:24:41.128068 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:41.128024    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:41.360022 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:41.359959    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:24:41.529784 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:41.529729    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8l26\" (UniqueName: \"kubernetes.io/projected/ec8ac321-2d7e-4f75-8082-a06152cd6712-kube-api-access-f8l26\") pod \"ec8ac321-2d7e-4f75-8082-a06152cd6712\" (UID: \"ec8ac321-2d7e-4f75-8082-a06152cd6712\") "
Jul 20 17:24:41.529784 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:41.529789    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec8ac321-2d7e-4f75-8082-a06152cd6712-openshift-service-ca\") pod \"ec8ac321-2d7e-4f75-8082-a06152cd6712\" (UID: \"ec8ac321-2d7e-4f75-8082-a06152cd6712\") "
Jul 20 17:24:41.530014 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:41.529828    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec8ac321-2d7e-4f75-8082-a06152cd6712-maas-api-tls\") pod \"ec8ac321-2d7e-4f75-8082-a06152cd6712\" (UID: \"ec8ac321-2d7e-4f75-8082-a06152cd6712\") "
Jul 20 17:24:41.530209 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:41.530184    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec8ac321-2d7e-4f75-8082-a06152cd6712-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ec8ac321-2d7e-4f75-8082-a06152cd6712" (UID: "ec8ac321-2d7e-4f75-8082-a06152cd6712"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:24:41.532072 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:41.532048    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec8ac321-2d7e-4f75-8082-a06152cd6712-kube-api-access-f8l26" (OuterVolumeSpecName: "kube-api-access-f8l26") pod "ec8ac321-2d7e-4f75-8082-a06152cd6712" (UID: "ec8ac321-2d7e-4f75-8082-a06152cd6712"). InnerVolumeSpecName "kube-api-access-f8l26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:24:41.532164 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:41.532145    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec8ac321-2d7e-4f75-8082-a06152cd6712-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ec8ac321-2d7e-4f75-8082-a06152cd6712" (UID: "ec8ac321-2d7e-4f75-8082-a06152cd6712"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:24:41.630767 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:41.630734    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8l26\" (UniqueName: \"kubernetes.io/projected/ec8ac321-2d7e-4f75-8082-a06152cd6712-kube-api-access-f8l26\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:24:41.630767 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:41.630767    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec8ac321-2d7e-4f75-8082-a06152cd6712-openshift-service-ca\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:24:41.630967 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:41.630777    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec8ac321-2d7e-4f75-8082-a06152cd6712-maas-api-tls\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:24:42.079191 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:42.079101    2568 generic.go:356] "Generic (PLEG): container finished" podID="ec8ac321-2d7e-4f75-8082-a06152cd6712" containerID="ec8664c8488b26df310cc7fad9868e10149787668a122bec0dadc5889e511b10" exitCode=0
Jul 20 17:24:42.079191 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:42.079144    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"
Jul 20 17:24:42.079191 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:42.079157    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575" event={"ID":"ec8ac321-2d7e-4f75-8082-a06152cd6712","Type":"ContainerDied","Data":"ec8664c8488b26df310cc7fad9868e10149787668a122bec0dadc5889e511b10"}
Jul 20 17:24:42.079456 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:42.079210    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575" event={"ID":"ec8ac321-2d7e-4f75-8082-a06152cd6712","Type":"ContainerDied","Data":"82d597b9f1bdda65a38b66a3039b43fb046c360c1d9eda46accf8100bdc29611"}
Jul 20 17:24:42.079456 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:42.079234    2568 scope.go:117] "RemoveContainer" containerID="ec8664c8488b26df310cc7fad9868e10149787668a122bec0dadc5889e511b10"
Jul 20 17:24:42.088443 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:42.088422    2568 scope.go:117] "RemoveContainer" containerID="ec8664c8488b26df310cc7fad9868e10149787668a122bec0dadc5889e511b10"
Jul 20 17:24:42.088689 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:24:42.088671    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec8664c8488b26df310cc7fad9868e10149787668a122bec0dadc5889e511b10\": container with ID starting with ec8664c8488b26df310cc7fad9868e10149787668a122bec0dadc5889e511b10 not found: ID does not exist" containerID="ec8664c8488b26df310cc7fad9868e10149787668a122bec0dadc5889e511b10"
Jul 20 17:24:42.088749 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:42.088702    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec8664c8488b26df310cc7fad9868e10149787668a122bec0dadc5889e511b10"} err="failed to get container status \"ec8664c8488b26df310cc7fad9868e10149787668a122bec0dadc5889e511b10\": rpc error: code = NotFound desc = could not find container \"ec8664c8488b26df310cc7fad9868e10149787668a122bec0dadc5889e511b10\": container with ID starting with ec8664c8488b26df310cc7fad9868e10149787668a122bec0dadc5889e511b10 not found: ID does not exist"
Jul 20 17:24:42.099466 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:42.099443    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"]
Jul 20 17:24:42.103200 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:42.103180    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c63878-5fc9cd7c96-j7575"]
Jul 20 17:24:42.897149 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:42.897112    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec8ac321-2d7e-4f75-8082-a06152cd6712" path="/var/lib/kubelet/pods/ec8ac321-2d7e-4f75-8082-a06152cd6712/volumes"
Jul 20 17:24:45.565342 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:45.565309    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:51.677555 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:51.677521    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:51.781124 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:51.781093    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:24:55.944155 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:24:55.944115    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:01.633449 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:01.633414    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:01.736991 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:01.736958    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:01.923114 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:01.923019    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:12.124659 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:12.124622    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:12.477566 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:12.477463    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:20.987867 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:20.987835    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:25:20.992074 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:20.992054    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:25:22.803277 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:22.803226    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:22.948449 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:22.948413    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:34.792885 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:34.792850    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:34.873855 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:34.873825    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:35.014131 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.014091    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"]
Jul 20 17:25:35.014679 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.014659    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec8ac321-2d7e-4f75-8082-a06152cd6712" containerName="maas-api"
Jul 20 17:25:35.014679 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.014678    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec8ac321-2d7e-4f75-8082-a06152cd6712" containerName="maas-api"
Jul 20 17:25:35.014865 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.014692    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9" containerName="maas-api"
Jul 20 17:25:35.014865 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.014700    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9" containerName="maas-api"
Jul 20 17:25:35.014865 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.014789    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec8ac321-2d7e-4f75-8082-a06152cd6712" containerName="maas-api"
Jul 20 17:25:35.014865 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.014805    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c742c3e2-a0d5-4b0a-8ba4-5cf2b52908c9" containerName="maas-api"
Jul 20 17:25:35.018105 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.018086    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:25:35.023411 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.023380    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-f5995d-serving-cert\""
Jul 20 17:25:35.033458 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.033430    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"]
Jul 20 17:25:35.062580 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.062483    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:35.199604 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.199577    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9wzw\" (UniqueName: \"kubernetes.io/projected/2bd62e71-5961-4339-a844-fd8e15b31685-kube-api-access-v9wzw\") pod \"maas-api-e2e-shared-a-f5995d-75564999c-lnlj9\" (UID: \"2bd62e71-5961-4339-a844-fd8e15b31685\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:25:35.199788 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.199635    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd62e71-5961-4339-a844-fd8e15b31685-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f5995d-75564999c-lnlj9\" (UID: \"2bd62e71-5961-4339-a844-fd8e15b31685\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:25:35.199788 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.199671    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd62e71-5961-4339-a844-fd8e15b31685-maas-api-tls\") pod \"maas-api-e2e-shared-a-f5995d-75564999c-lnlj9\" (UID: \"2bd62e71-5961-4339-a844-fd8e15b31685\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:25:35.300861 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.300825    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd62e71-5961-4339-a844-fd8e15b31685-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f5995d-75564999c-lnlj9\" (UID: \"2bd62e71-5961-4339-a844-fd8e15b31685\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:25:35.301058 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.300873    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd62e71-5961-4339-a844-fd8e15b31685-maas-api-tls\") pod \"maas-api-e2e-shared-a-f5995d-75564999c-lnlj9\" (UID: \"2bd62e71-5961-4339-a844-fd8e15b31685\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:25:35.301058 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.300955    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9wzw\" (UniqueName: \"kubernetes.io/projected/2bd62e71-5961-4339-a844-fd8e15b31685-kube-api-access-v9wzw\") pod \"maas-api-e2e-shared-a-f5995d-75564999c-lnlj9\" (UID: \"2bd62e71-5961-4339-a844-fd8e15b31685\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:25:35.301499 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.301472    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd62e71-5961-4339-a844-fd8e15b31685-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f5995d-75564999c-lnlj9\" (UID: \"2bd62e71-5961-4339-a844-fd8e15b31685\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:25:35.303656 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.303625    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd62e71-5961-4339-a844-fd8e15b31685-maas-api-tls\") pod \"maas-api-e2e-shared-a-f5995d-75564999c-lnlj9\" (UID: \"2bd62e71-5961-4339-a844-fd8e15b31685\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:25:35.312090 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.312066    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9wzw\" (UniqueName: \"kubernetes.io/projected/2bd62e71-5961-4339-a844-fd8e15b31685-kube-api-access-v9wzw\") pod \"maas-api-e2e-shared-a-f5995d-75564999c-lnlj9\" (UID: \"2bd62e71-5961-4339-a844-fd8e15b31685\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:25:35.331104 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.331039    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:25:35.484614 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.484590    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"]
Jul 20 17:25:35.486404 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:25:35.486378    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd62e71_5961_4339_a844_fd8e15b31685.slice/crio-a46dd4d4188c3d5280d0f566d72859f367c54c4bd4a2baae60e62326945d4915 WatchSource:0}: Error finding container a46dd4d4188c3d5280d0f566d72859f367c54c4bd4a2baae60e62326945d4915: Status 404 returned error can't find the container with id a46dd4d4188c3d5280d0f566d72859f367c54c4bd4a2baae60e62326945d4915
Jul 20 17:25:35.604176 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.604090    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:35.703896 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:35.703839    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:36.339615 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:36.339528    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9" event={"ID":"2bd62e71-5961-4339-a844-fd8e15b31685","Type":"ContainerStarted","Data":"53cc1eb86c2c21d36f4ac9941a53d1eb67f7371a0020c9dc54b7d5a523357e0c"}
Jul 20 17:25:36.339615 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:36.339581    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9" event={"ID":"2bd62e71-5961-4339-a844-fd8e15b31685","Type":"ContainerStarted","Data":"a46dd4d4188c3d5280d0f566d72859f367c54c4bd4a2baae60e62326945d4915"}
Jul 20 17:25:36.339615 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:36.339612    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:25:36.360456 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:36.360394    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9" podStartSLOduration=1.9692774339999999 podStartE2EDuration="2.360374125s" totalImagesPullingTime="391.096691ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:25:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:25:36.357271693 +0000 UTC m=+2415.905127761" watchObservedRunningTime="2026-07-20 17:25:36.360374125 +0000 UTC m=+2415.908230191"
Jul 20 17:25:39.215678 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:39.215645    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:42.350441 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:42.350350    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:25:49.612199 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:49.612148    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:49.969354 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:49.969315    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:50.070172 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:50.070141    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:53.083819 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.083780    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:53.165995 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.165962    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"]
Jul 20 17:25:53.170201 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.170179    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:25:53.173359 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.173340    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-a24b97-serving-cert\""
Jul 20 17:25:53.175392 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.175370    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:53.182400 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.182376    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"]
Jul 20 17:25:53.251567 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.251532    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:53.259715 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.259688    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ecce5e-8715-4762-bcf5-f62264b70a64-maas-api-tls\") pod \"maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg\" (UID: \"55ecce5e-8715-4762-bcf5-f62264b70a64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:25:53.259852 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.259732    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ecce5e-8715-4762-bcf5-f62264b70a64-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg\" (UID: \"55ecce5e-8715-4762-bcf5-f62264b70a64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:25:53.259852 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.259841    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmftx\" (UniqueName: \"kubernetes.io/projected/55ecce5e-8715-4762-bcf5-f62264b70a64-kube-api-access-wmftx\") pod \"maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg\" (UID: \"55ecce5e-8715-4762-bcf5-f62264b70a64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:25:53.358765 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.358663    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:53.360753 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.360724    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmftx\" (UniqueName: \"kubernetes.io/projected/55ecce5e-8715-4762-bcf5-f62264b70a64-kube-api-access-wmftx\") pod \"maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg\" (UID: \"55ecce5e-8715-4762-bcf5-f62264b70a64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:25:53.360903 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.360787    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ecce5e-8715-4762-bcf5-f62264b70a64-maas-api-tls\") pod \"maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg\" (UID: \"55ecce5e-8715-4762-bcf5-f62264b70a64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:25:53.360903 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.360818    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ecce5e-8715-4762-bcf5-f62264b70a64-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg\" (UID: \"55ecce5e-8715-4762-bcf5-f62264b70a64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:25:53.361527 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.361500    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ecce5e-8715-4762-bcf5-f62264b70a64-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg\" (UID: \"55ecce5e-8715-4762-bcf5-f62264b70a64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:25:53.363435 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.363407    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ecce5e-8715-4762-bcf5-f62264b70a64-maas-api-tls\") pod \"maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg\" (UID: \"55ecce5e-8715-4762-bcf5-f62264b70a64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:25:53.372420 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.372396    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmftx\" (UniqueName: \"kubernetes.io/projected/55ecce5e-8715-4762-bcf5-f62264b70a64-kube-api-access-wmftx\") pod \"maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg\" (UID: \"55ecce5e-8715-4762-bcf5-f62264b70a64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:25:53.482633 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.482603    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:25:53.623858 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.623834    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"]
Jul 20 17:25:53.625583 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:25:53.625552    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55ecce5e_8715_4762_bcf5_f62264b70a64.slice/crio-2dbdfc3e3be6f8d898080d216ff62e4ad832340b1949d2d0fe6f5cd0bab8259c WatchSource:0}: Error finding container 2dbdfc3e3be6f8d898080d216ff62e4ad832340b1949d2d0fe6f5cd0bab8259c: Status 404 returned error can't find the container with id 2dbdfc3e3be6f8d898080d216ff62e4ad832340b1949d2d0fe6f5cd0bab8259c
Jul 20 17:25:53.772858 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.772813    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:53.872124 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:53.872083    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:54.437159 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:54.437125    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg" event={"ID":"55ecce5e-8715-4762-bcf5-f62264b70a64","Type":"ContainerStarted","Data":"749cea6988c74859332ad3dfe031d80f1e699f6cc624a5f99145216c27ea4758"}
Jul 20 17:25:54.437594 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:54.437173    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg" event={"ID":"55ecce5e-8715-4762-bcf5-f62264b70a64","Type":"ContainerStarted","Data":"2dbdfc3e3be6f8d898080d216ff62e4ad832340b1949d2d0fe6f5cd0bab8259c"}
Jul 20 17:25:54.437594 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:54.437228    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:25:54.461435 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:54.461388    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg" podStartSLOduration=1.0789261589999999 podStartE2EDuration="1.461373219s" totalImagesPullingTime="382.44706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:25:54.460148612 +0000 UTC m=+2434.008004678" watchObservedRunningTime="2026-07-20 17:25:54.461373219 +0000 UTC m=+2434.009229286"
Jul 20 17:25:57.567778 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:57.567740    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:25:57.667762 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:25:57.667725    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:26:00.448252 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:00.448217    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:26:07.498702 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:07.498654    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:26:14.408362 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.408328    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"]
Jul 20 17:26:14.412534 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.412513    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.416465 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.416440    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f5995d\"/\"kube-root-ca.crt\""
Jul 20 17:26:14.416570 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.416480    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f5995d\"/\"openshift-service-ca.crt\""
Jul 20 17:26:14.416570 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.416489    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f5995d\"/\"auth-test-model-f5995d-kserve-self-signed-certs\""
Jul 20 17:26:14.416570 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.416509    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f5995d\"/\"default-dockercfg-n8l7n\""
Jul 20 17:26:14.420039 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.420011    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"]
Jul 20 17:26:14.467258 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.467201    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:26:14.545307 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.545275    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/387f6b37-a620-4542-ab37-24dbd4f02b58-tls-certs\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.545493 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.545345    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-home\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.545493 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.545366    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-dshm\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.545493 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.545397    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-model-cache\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.545626 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.545503    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-tmp-dir\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.545626 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.545564    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59z4p\" (UniqueName: \"kubernetes.io/projected/387f6b37-a620-4542-ab37-24dbd4f02b58-kube-api-access-59z4p\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.646780 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.646746    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/387f6b37-a620-4542-ab37-24dbd4f02b58-tls-certs\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.646977 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.646788    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-home\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.646977 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.646814    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-dshm\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.646977 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.646932    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-model-cache\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.647162 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.647000    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-tmp-dir\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.647162 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.647043    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59z4p\" (UniqueName: \"kubernetes.io/projected/387f6b37-a620-4542-ab37-24dbd4f02b58-kube-api-access-59z4p\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.647301 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.647230    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-home\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.647348 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.647322    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-model-cache\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.647535 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.647514    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-tmp-dir\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.649515 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.649495    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-dshm\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.649750 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.649733    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/387f6b37-a620-4542-ab37-24dbd4f02b58-tls-certs\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.655156 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.655134    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59z4p\" (UniqueName: \"kubernetes.io/projected/387f6b37-a620-4542-ab37-24dbd4f02b58-kube-api-access-59z4p\") pod \"auth-test-model-f5995d-kserve-6696cc6899-6bhqj\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") " pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.666508 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.666438    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:26:14.726200 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.726167    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:14.768998 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.768433    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:26:14.864938 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:14.864914    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"]
Jul 20 17:26:14.867057 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:26:14.867011    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod387f6b37_a620_4542_ab37_24dbd4f02b58.slice/crio-9d4c121c2aab26f54e560b7e82ee1854428546d4160c3d9eb2140ea151de2b76 WatchSource:0}: Error finding container 9d4c121c2aab26f54e560b7e82ee1854428546d4160c3d9eb2140ea151de2b76: Status 404 returned error can't find the container with id 9d4c121c2aab26f54e560b7e82ee1854428546d4160c3d9eb2140ea151de2b76
Jul 20 17:26:15.543579 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:15.543547    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj" event={"ID":"387f6b37-a620-4542-ab37-24dbd4f02b58","Type":"ContainerStarted","Data":"3b2b62ef05a7f92c1dcf5038b5c7783df3232dfec075b92614e37af22700e42e"}
Jul 20 17:26:15.543579 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:15.543589    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj" event={"ID":"387f6b37-a620-4542-ab37-24dbd4f02b58","Type":"ContainerStarted","Data":"9d4c121c2aab26f54e560b7e82ee1854428546d4160c3d9eb2140ea151de2b76"}
Jul 20 17:26:15.587748 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:15.587692    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj" podStartSLOduration=1.587677052 podStartE2EDuration="1.587677052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:26:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:26:15.586713931 +0000 UTC m=+2455.134569996" watchObservedRunningTime="2026-07-20 17:26:15.587677052 +0000 UTC m=+2455.135533136"
Jul 20 17:26:24.727330 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:24.727289    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:24.727840 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:24.727343    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:24.739733 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:24.739709    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:25.604411 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:25.604383    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:26:26.874067 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:26.874032    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:26:26.926898 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:26:26.926867    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:27:02.755203 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:27:02.755169    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:27:04.802460 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:27:04.802421    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:29:12.004974 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:12.004944    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"]
Jul 20 17:29:12.054310 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:12.054261    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:29:13.199603 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:13.199567    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:29:24.804150 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:24.804118    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"]
Jul 20 17:29:24.807684 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:24.807665    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:24.810541 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:24.810522    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f5995d\"/\"sub-test-model-f5995d-kserve-self-signed-certs\""
Jul 20 17:29:24.818684 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:24.818659    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"]
Jul 20 17:29:24.837063 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:24.837027    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:29:24.918530 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:24.918497    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxzrl\" (UniqueName: \"kubernetes.io/projected/c8ac2f41-b499-4134-a650-0d6158c04098-kube-api-access-lxzrl\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:24.918530 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:24.918537    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8ac2f41-b499-4134-a650-0d6158c04098-tls-certs\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:24.918768 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:24.918572    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-dshm\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:24.918768 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:24.918656    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-model-cache\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:24.918768 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:24.918688    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-home\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:24.918768 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:24.918711    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-tmp-dir\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.019624 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.019592    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxzrl\" (UniqueName: \"kubernetes.io/projected/c8ac2f41-b499-4134-a650-0d6158c04098-kube-api-access-lxzrl\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.019823 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.019632    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8ac2f41-b499-4134-a650-0d6158c04098-tls-certs\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.019823 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.019661    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-dshm\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.019823 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.019745    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-model-cache\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.019823 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.019777    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-home\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.019823 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.019800    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-tmp-dir\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.020218 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.020185    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-model-cache\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.020218 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.020199    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-home\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.020400 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.020231    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-tmp-dir\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.022226 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.022202    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-dshm\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.022419 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.022401    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8ac2f41-b499-4134-a650-0d6158c04098-tls-certs\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.030189 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.030164    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxzrl\" (UniqueName: \"kubernetes.io/projected/c8ac2f41-b499-4134-a650-0d6158c04098-kube-api-access-lxzrl\") pod \"sub-test-model-f5995d-kserve-fb5df75ff-8lv92\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") " pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.117211 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.117137    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:25.258052 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.258025    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"]
Jul 20 17:29:25.260806 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:29:25.260775    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8ac2f41_b499_4134_a650_0d6158c04098.slice/crio-f9738e27eafe0f26de8c7c8628bf0ade54cbd21778f0501076f934213cd3c2d7 WatchSource:0}: Error finding container f9738e27eafe0f26de8c7c8628bf0ade54cbd21778f0501076f934213cd3c2d7: Status 404 returned error can't find the container with id f9738e27eafe0f26de8c7c8628bf0ade54cbd21778f0501076f934213cd3c2d7
Jul 20 17:29:25.488192 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.488159    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92" event={"ID":"c8ac2f41-b499-4134-a650-0d6158c04098","Type":"ContainerStarted","Data":"fc845e01bf89397bb5cd0ce543c0d65b34d1b11b15895692ebac81d032efbd48"}
Jul 20 17:29:25.488192 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.488198    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92" event={"ID":"c8ac2f41-b499-4134-a650-0d6158c04098","Type":"ContainerStarted","Data":"f9738e27eafe0f26de8c7c8628bf0ade54cbd21778f0501076f934213cd3c2d7"}
Jul 20 17:29:25.506548 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:25.506495    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92" podStartSLOduration=1.5064803850000001 podStartE2EDuration="1.506480385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:29:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:29:25.504309174 +0000 UTC m=+2645.052165240" watchObservedRunningTime="2026-07-20 17:29:25.506480385 +0000 UTC m=+2645.054336451"
Jul 20 17:29:27.025453 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.025411    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj" podUID="387f6b37-a620-4542-ab37-24dbd4f02b58" containerName="main" containerID="cri-o://3b2b62ef05a7f92c1dcf5038b5c7783df3232dfec075b92614e37af22700e42e" gracePeriod=45
Jul 20 17:29:27.279251 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.279167    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:29:27.344087 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.344055    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/387f6b37-a620-4542-ab37-24dbd4f02b58-tls-certs\") pod \"387f6b37-a620-4542-ab37-24dbd4f02b58\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") "
Jul 20 17:29:27.344087 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.344090    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-home\") pod \"387f6b37-a620-4542-ab37-24dbd4f02b58\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") "
Jul 20 17:29:27.344359 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.344160    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-model-cache\") pod \"387f6b37-a620-4542-ab37-24dbd4f02b58\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") "
Jul 20 17:29:27.344359 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.344213    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-dshm\") pod \"387f6b37-a620-4542-ab37-24dbd4f02b58\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") "
Jul 20 17:29:27.344359 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.344261    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59z4p\" (UniqueName: \"kubernetes.io/projected/387f6b37-a620-4542-ab37-24dbd4f02b58-kube-api-access-59z4p\") pod \"387f6b37-a620-4542-ab37-24dbd4f02b58\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") "
Jul 20 17:29:27.344359 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.344297    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-tmp-dir\") pod \"387f6b37-a620-4542-ab37-24dbd4f02b58\" (UID: \"387f6b37-a620-4542-ab37-24dbd4f02b58\") "
Jul 20 17:29:27.344584 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.344468    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-model-cache" (OuterVolumeSpecName: "model-cache") pod "387f6b37-a620-4542-ab37-24dbd4f02b58" (UID: "387f6b37-a620-4542-ab37-24dbd4f02b58"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:29:27.344584 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.344480    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-home" (OuterVolumeSpecName: "home") pod "387f6b37-a620-4542-ab37-24dbd4f02b58" (UID: "387f6b37-a620-4542-ab37-24dbd4f02b58"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:29:27.344584 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.344580    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-home\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:29:27.344725 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.344594    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-model-cache\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:29:27.344725 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.344660    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "387f6b37-a620-4542-ab37-24dbd4f02b58" (UID: "387f6b37-a620-4542-ab37-24dbd4f02b58"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:29:27.346819 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.346790    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/387f6b37-a620-4542-ab37-24dbd4f02b58-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "387f6b37-a620-4542-ab37-24dbd4f02b58" (UID: "387f6b37-a620-4542-ab37-24dbd4f02b58"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:29:27.346963 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.346819    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-dshm" (OuterVolumeSpecName: "dshm") pod "387f6b37-a620-4542-ab37-24dbd4f02b58" (UID: "387f6b37-a620-4542-ab37-24dbd4f02b58"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:29:27.347229 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.347207    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/387f6b37-a620-4542-ab37-24dbd4f02b58-kube-api-access-59z4p" (OuterVolumeSpecName: "kube-api-access-59z4p") pod "387f6b37-a620-4542-ab37-24dbd4f02b58" (UID: "387f6b37-a620-4542-ab37-24dbd4f02b58"). InnerVolumeSpecName "kube-api-access-59z4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:29:27.445325 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.445284    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/387f6b37-a620-4542-ab37-24dbd4f02b58-tls-certs\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:29:27.445325 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.445317    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-dshm\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:29:27.445325 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.445331    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59z4p\" (UniqueName: \"kubernetes.io/projected/387f6b37-a620-4542-ab37-24dbd4f02b58-kube-api-access-59z4p\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:29:27.445618 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.445343    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/387f6b37-a620-4542-ab37-24dbd4f02b58-tmp-dir\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:29:27.499322 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.499284    2568 generic.go:356] "Generic (PLEG): container finished" podID="387f6b37-a620-4542-ab37-24dbd4f02b58" containerID="3b2b62ef05a7f92c1dcf5038b5c7783df3232dfec075b92614e37af22700e42e" exitCode=0
Jul 20 17:29:27.499477 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.499325    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"
Jul 20 17:29:27.499477 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.499376    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj" event={"ID":"387f6b37-a620-4542-ab37-24dbd4f02b58","Type":"ContainerDied","Data":"3b2b62ef05a7f92c1dcf5038b5c7783df3232dfec075b92614e37af22700e42e"}
Jul 20 17:29:27.499477 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.499412    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj" event={"ID":"387f6b37-a620-4542-ab37-24dbd4f02b58","Type":"ContainerDied","Data":"9d4c121c2aab26f54e560b7e82ee1854428546d4160c3d9eb2140ea151de2b76"}
Jul 20 17:29:27.499477 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.499434    2568 scope.go:117] "RemoveContainer" containerID="3b2b62ef05a7f92c1dcf5038b5c7783df3232dfec075b92614e37af22700e42e"
Jul 20 17:29:27.511155 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.511126    2568 scope.go:117] "RemoveContainer" containerID="3b2b62ef05a7f92c1dcf5038b5c7783df3232dfec075b92614e37af22700e42e"
Jul 20 17:29:27.511715 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:29:27.511695    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b2b62ef05a7f92c1dcf5038b5c7783df3232dfec075b92614e37af22700e42e\": container with ID starting with 3b2b62ef05a7f92c1dcf5038b5c7783df3232dfec075b92614e37af22700e42e not found: ID does not exist" containerID="3b2b62ef05a7f92c1dcf5038b5c7783df3232dfec075b92614e37af22700e42e"
Jul 20 17:29:27.511807 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.511722    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b2b62ef05a7f92c1dcf5038b5c7783df3232dfec075b92614e37af22700e42e"} err="failed to get container status \"3b2b62ef05a7f92c1dcf5038b5c7783df3232dfec075b92614e37af22700e42e\": rpc error: code = NotFound desc = could not find container \"3b2b62ef05a7f92c1dcf5038b5c7783df3232dfec075b92614e37af22700e42e\": container with ID starting with 3b2b62ef05a7f92c1dcf5038b5c7783df3232dfec075b92614e37af22700e42e not found: ID does not exist"
Jul 20 17:29:27.521685 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.521596    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"]
Jul 20 17:29:27.525816 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:27.525791    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/auth-test-model-f5995d-kserve-6696cc6899-6bhqj"]
Jul 20 17:29:28.897510 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:28.897475    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="387f6b37-a620-4542-ab37-24dbd4f02b58" path="/var/lib/kubelet/pods/387f6b37-a620-4542-ab37-24dbd4f02b58/volumes"
Jul 20 17:29:30.284115 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:30.284081    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:29:35.117712 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:35.117679    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:35.118085 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:35.117730    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:35.131922 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:35.131896    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:35.555327 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:35.555301    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:29:49.076856 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:49.076816    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:29:52.488974 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:52.488942    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:29:52.588642 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:29:52.588606    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:30:21.016292 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:21.016190    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:30:21.022153 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:21.022132    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-44gfk_75b664df-5e35-4825-a3ae-e8310a71b97e/ovn-acl-logging/0.log"
Jul 20 17:30:24.227173 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:24.227140    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"]
Jul 20 17:30:24.512458 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:24.512370    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:30:25.224130 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:25.224097    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:30:38.586619 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.586583    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"]
Jul 20 17:30:38.587191 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.587169    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="387f6b37-a620-4542-ab37-24dbd4f02b58" containerName="main"
Jul 20 17:30:38.587399 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.587378    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="387f6b37-a620-4542-ab37-24dbd4f02b58" containerName="main"
Jul 20 17:30:38.587547 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.587535    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="387f6b37-a620-4542-ab37-24dbd4f02b58" containerName="main"
Jul 20 17:30:38.590946 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.590926    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.593635 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.593615    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f5995d\"/\"rate-test-model-f5995d-kserve-self-signed-certs\""
Jul 20 17:30:38.600324 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.599935    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"]
Jul 20 17:30:38.620403 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.620371    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:30:38.677068 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.677034    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-home\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.677068 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.677071    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-model-cache\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.677292 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.677100    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-tmp-dir\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.677292 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.677188    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d57bf203-9e00-41b3-8956-7ad7239c2981-tls-certs\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.677292 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.677279    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-dshm\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.677394 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.677323    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jdgx\" (UniqueName: \"kubernetes.io/projected/d57bf203-9e00-41b3-8956-7ad7239c2981-kube-api-access-4jdgx\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.778780 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.778740    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-dshm\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.778966 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.778803    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jdgx\" (UniqueName: \"kubernetes.io/projected/d57bf203-9e00-41b3-8956-7ad7239c2981-kube-api-access-4jdgx\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.778966 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.778853    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-home\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.778966 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.778881    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-model-cache\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.778966 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.778908    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-tmp-dir\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.778966 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.778941    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d57bf203-9e00-41b3-8956-7ad7239c2981-tls-certs\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.779334 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.779306    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-home\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.779334 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.779320    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-tmp-dir\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.779460 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.779351    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-model-cache\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.781185 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.781165    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-dshm\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.781543 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.781524    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d57bf203-9e00-41b3-8956-7ad7239c2981-tls-certs\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.787351 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.787327    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jdgx\" (UniqueName: \"kubernetes.io/projected/d57bf203-9e00-41b3-8956-7ad7239c2981-kube-api-access-4jdgx\") pod \"rate-test-model-f5995d-kserve-54f549c79f-5zl97\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") " pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:38.904719 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:38.904686    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:39.040212 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.040184    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"]
Jul 20 17:30:39.042543 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:30:39.042491    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd57bf203_9e00_41b3_8956_7ad7239c2981.slice/crio-96ec4296fb83fc2b3115b5b353dc0a8aa83b20bf64b32fd0277f1fc7b9a5ef21 WatchSource:0}: Error finding container 96ec4296fb83fc2b3115b5b353dc0a8aa83b20bf64b32fd0277f1fc7b9a5ef21: Status 404 returned error can't find the container with id 96ec4296fb83fc2b3115b5b353dc0a8aa83b20bf64b32fd0277f1fc7b9a5ef21
Jul 20 17:30:39.251381 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.251289    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92" podUID="c8ac2f41-b499-4134-a650-0d6158c04098" containerName="main" containerID="cri-o://fc845e01bf89397bb5cd0ce543c0d65b34d1b11b15895692ebac81d032efbd48" gracePeriod=45
Jul 20 17:30:39.500321 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.500295    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:30:39.586937 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.586836    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8ac2f41-b499-4134-a650-0d6158c04098-tls-certs\") pod \"c8ac2f41-b499-4134-a650-0d6158c04098\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") "
Jul 20 17:30:39.586937 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.586887    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-model-cache\") pod \"c8ac2f41-b499-4134-a650-0d6158c04098\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") "
Jul 20 17:30:39.586937 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.586904    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-tmp-dir\") pod \"c8ac2f41-b499-4134-a650-0d6158c04098\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") "
Jul 20 17:30:39.586937 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.586937    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxzrl\" (UniqueName: \"kubernetes.io/projected/c8ac2f41-b499-4134-a650-0d6158c04098-kube-api-access-lxzrl\") pod \"c8ac2f41-b499-4134-a650-0d6158c04098\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") "
Jul 20 17:30:39.587523 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.586963    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-dshm\") pod \"c8ac2f41-b499-4134-a650-0d6158c04098\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") "
Jul 20 17:30:39.587523 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.587082    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-home\") pod \"c8ac2f41-b499-4134-a650-0d6158c04098\" (UID: \"c8ac2f41-b499-4134-a650-0d6158c04098\") "
Jul 20 17:30:39.587523 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.587140    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-model-cache" (OuterVolumeSpecName: "model-cache") pod "c8ac2f41-b499-4134-a650-0d6158c04098" (UID: "c8ac2f41-b499-4134-a650-0d6158c04098"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:30:39.587523 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.587176    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c8ac2f41-b499-4134-a650-0d6158c04098" (UID: "c8ac2f41-b499-4134-a650-0d6158c04098"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:30:39.587523 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.587385    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-model-cache\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:30:39.587523 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.587401    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-tmp-dir\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:30:39.587523 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.587412    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-home" (OuterVolumeSpecName: "home") pod "c8ac2f41-b499-4134-a650-0d6158c04098" (UID: "c8ac2f41-b499-4134-a650-0d6158c04098"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:30:39.589272 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.589235    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8ac2f41-b499-4134-a650-0d6158c04098-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c8ac2f41-b499-4134-a650-0d6158c04098" (UID: "c8ac2f41-b499-4134-a650-0d6158c04098"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:30:39.589272 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.589258    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-dshm" (OuterVolumeSpecName: "dshm") pod "c8ac2f41-b499-4134-a650-0d6158c04098" (UID: "c8ac2f41-b499-4134-a650-0d6158c04098"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:30:39.589399 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.589298    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8ac2f41-b499-4134-a650-0d6158c04098-kube-api-access-lxzrl" (OuterVolumeSpecName: "kube-api-access-lxzrl") pod "c8ac2f41-b499-4134-a650-0d6158c04098" (UID: "c8ac2f41-b499-4134-a650-0d6158c04098"). InnerVolumeSpecName "kube-api-access-lxzrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:30:39.688542 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.688500    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-home\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:30:39.688542 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.688537    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8ac2f41-b499-4134-a650-0d6158c04098-tls-certs\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:30:39.688542 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.688548    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxzrl\" (UniqueName: \"kubernetes.io/projected/c8ac2f41-b499-4134-a650-0d6158c04098-kube-api-access-lxzrl\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:30:39.688542 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.688557    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8ac2f41-b499-4134-a650-0d6158c04098-dshm\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:30:39.868179 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.868099    2568 generic.go:356] "Generic (PLEG): container finished" podID="c8ac2f41-b499-4134-a650-0d6158c04098" containerID="fc845e01bf89397bb5cd0ce543c0d65b34d1b11b15895692ebac81d032efbd48" exitCode=0
Jul 20 17:30:39.868179 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.868146    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"
Jul 20 17:30:39.868407 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.868184    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92" event={"ID":"c8ac2f41-b499-4134-a650-0d6158c04098","Type":"ContainerDied","Data":"fc845e01bf89397bb5cd0ce543c0d65b34d1b11b15895692ebac81d032efbd48"}
Jul 20 17:30:39.868407 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.868230    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92" event={"ID":"c8ac2f41-b499-4134-a650-0d6158c04098","Type":"ContainerDied","Data":"f9738e27eafe0f26de8c7c8628bf0ade54cbd21778f0501076f934213cd3c2d7"}
Jul 20 17:30:39.868407 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.868279    2568 scope.go:117] "RemoveContainer" containerID="fc845e01bf89397bb5cd0ce543c0d65b34d1b11b15895692ebac81d032efbd48"
Jul 20 17:30:39.869823 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.869799    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97" event={"ID":"d57bf203-9e00-41b3-8956-7ad7239c2981","Type":"ContainerStarted","Data":"8e655781785e281ac275078184baf60ca83f69431b30df8a5e97a08478a9f2f9"}
Jul 20 17:30:39.869910 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.869836    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97" event={"ID":"d57bf203-9e00-41b3-8956-7ad7239c2981","Type":"ContainerStarted","Data":"96ec4296fb83fc2b3115b5b353dc0a8aa83b20bf64b32fd0277f1fc7b9a5ef21"}
Jul 20 17:30:39.877333 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.877302    2568 scope.go:117] "RemoveContainer" containerID="fc845e01bf89397bb5cd0ce543c0d65b34d1b11b15895692ebac81d032efbd48"
Jul 20 17:30:39.877578 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:30:39.877560    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc845e01bf89397bb5cd0ce543c0d65b34d1b11b15895692ebac81d032efbd48\": container with ID starting with fc845e01bf89397bb5cd0ce543c0d65b34d1b11b15895692ebac81d032efbd48 not found: ID does not exist" containerID="fc845e01bf89397bb5cd0ce543c0d65b34d1b11b15895692ebac81d032efbd48"
Jul 20 17:30:39.877637 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.877585    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc845e01bf89397bb5cd0ce543c0d65b34d1b11b15895692ebac81d032efbd48"} err="failed to get container status \"fc845e01bf89397bb5cd0ce543c0d65b34d1b11b15895692ebac81d032efbd48\": rpc error: code = NotFound desc = could not find container \"fc845e01bf89397bb5cd0ce543c0d65b34d1b11b15895692ebac81d032efbd48\": container with ID starting with fc845e01bf89397bb5cd0ce543c0d65b34d1b11b15895692ebac81d032efbd48 not found: ID does not exist"
Jul 20 17:30:39.892208 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.892157    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97" podStartSLOduration=1.892145369 podStartE2EDuration="1.892145369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:30:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:30:39.88956537 +0000 UTC m=+2719.437421438" watchObservedRunningTime="2026-07-20 17:30:39.892145369 +0000 UTC m=+2719.440001434"
Jul 20 17:30:39.903138 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.903114    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"]
Jul 20 17:30:39.906856 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:39.906837    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/sub-test-model-f5995d-kserve-fb5df75ff-8lv92"]
Jul 20 17:30:40.896552 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:40.896520    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8ac2f41-b499-4134-a650-0d6158c04098" path="/var/lib/kubelet/pods/c8ac2f41-b499-4134-a650-0d6158c04098/volumes"
Jul 20 17:30:48.905770 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:48.905742    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:48.906217 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:48.905791    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:48.918719 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:48.918688    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:49.928834 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:49.928807    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:30:58.444876 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:58.444838    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:30:58.545598 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:30:58.545564    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:31:13.089767 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:31:13.089732    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:31:15.449621 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:31:15.449579    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:31:15.546518 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:31:15.546478    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:31:56.818351 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:31:56.818312    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"]
Jul 20 17:31:56.837775 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:31:56.837743    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:31:57.949486 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:31:57.949450    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:07.540336 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:07.540303    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:07.596189 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:07.596156    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:11.843130 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:11.843074    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97" podUID="d57bf203-9e00-41b3-8956-7ad7239c2981" containerName="main" containerID="cri-o://8e655781785e281ac275078184baf60ca83f69431b30df8a5e97a08478a9f2f9" gracePeriod=45
Jul 20 17:32:12.096578 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.096521    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:32:12.211575 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.211543    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-tmp-dir\") pod \"d57bf203-9e00-41b3-8956-7ad7239c2981\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") "
Jul 20 17:32:12.211575 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.211577    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jdgx\" (UniqueName: \"kubernetes.io/projected/d57bf203-9e00-41b3-8956-7ad7239c2981-kube-api-access-4jdgx\") pod \"d57bf203-9e00-41b3-8956-7ad7239c2981\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") "
Jul 20 17:32:12.211794 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.211607    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-home\") pod \"d57bf203-9e00-41b3-8956-7ad7239c2981\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") "
Jul 20 17:32:12.211794 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.211641    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d57bf203-9e00-41b3-8956-7ad7239c2981-tls-certs\") pod \"d57bf203-9e00-41b3-8956-7ad7239c2981\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") "
Jul 20 17:32:12.211794 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.211660    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-model-cache\") pod \"d57bf203-9e00-41b3-8956-7ad7239c2981\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") "
Jul 20 17:32:12.211794 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.211715    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-dshm\") pod \"d57bf203-9e00-41b3-8956-7ad7239c2981\" (UID: \"d57bf203-9e00-41b3-8956-7ad7239c2981\") "
Jul 20 17:32:12.212007 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.211825    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-home" (OuterVolumeSpecName: "home") pod "d57bf203-9e00-41b3-8956-7ad7239c2981" (UID: "d57bf203-9e00-41b3-8956-7ad7239c2981"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:32:12.212007 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.211850    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d57bf203-9e00-41b3-8956-7ad7239c2981" (UID: "d57bf203-9e00-41b3-8956-7ad7239c2981"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:32:12.212007 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.211973    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-model-cache" (OuterVolumeSpecName: "model-cache") pod "d57bf203-9e00-41b3-8956-7ad7239c2981" (UID: "d57bf203-9e00-41b3-8956-7ad7239c2981"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:32:12.212007 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.211996    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-tmp-dir\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:32:12.212007 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.212009    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-home\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:32:12.213874 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.213839    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-dshm" (OuterVolumeSpecName: "dshm") pod "d57bf203-9e00-41b3-8956-7ad7239c2981" (UID: "d57bf203-9e00-41b3-8956-7ad7239c2981"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:32:12.213981 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.213937    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d57bf203-9e00-41b3-8956-7ad7239c2981-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d57bf203-9e00-41b3-8956-7ad7239c2981" (UID: "d57bf203-9e00-41b3-8956-7ad7239c2981"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:32:12.213981 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.213942    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d57bf203-9e00-41b3-8956-7ad7239c2981-kube-api-access-4jdgx" (OuterVolumeSpecName: "kube-api-access-4jdgx") pod "d57bf203-9e00-41b3-8956-7ad7239c2981" (UID: "d57bf203-9e00-41b3-8956-7ad7239c2981"). InnerVolumeSpecName "kube-api-access-4jdgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:32:12.313296 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.313234    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jdgx\" (UniqueName: \"kubernetes.io/projected/d57bf203-9e00-41b3-8956-7ad7239c2981-kube-api-access-4jdgx\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:32:12.313296 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.313293    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d57bf203-9e00-41b3-8956-7ad7239c2981-tls-certs\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:32:12.313513 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.313309    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-model-cache\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:32:12.313513 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.313317    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d57bf203-9e00-41b3-8956-7ad7239c2981-dshm\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:32:12.318389 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.318364    2568 generic.go:356] "Generic (PLEG): container finished" podID="d57bf203-9e00-41b3-8956-7ad7239c2981" containerID="8e655781785e281ac275078184baf60ca83f69431b30df8a5e97a08478a9f2f9" exitCode=0
Jul 20 17:32:12.318513 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.318405    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"
Jul 20 17:32:12.318513 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.318429    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97" event={"ID":"d57bf203-9e00-41b3-8956-7ad7239c2981","Type":"ContainerDied","Data":"8e655781785e281ac275078184baf60ca83f69431b30df8a5e97a08478a9f2f9"}
Jul 20 17:32:12.318513 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.318459    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97" event={"ID":"d57bf203-9e00-41b3-8956-7ad7239c2981","Type":"ContainerDied","Data":"96ec4296fb83fc2b3115b5b353dc0a8aa83b20bf64b32fd0277f1fc7b9a5ef21"}
Jul 20 17:32:12.318513 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.318480    2568 scope.go:117] "RemoveContainer" containerID="8e655781785e281ac275078184baf60ca83f69431b30df8a5e97a08478a9f2f9"
Jul 20 17:32:12.328375 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.328326    2568 scope.go:117] "RemoveContainer" containerID="8e655781785e281ac275078184baf60ca83f69431b30df8a5e97a08478a9f2f9"
Jul 20 17:32:12.328572 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:32:12.328551    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e655781785e281ac275078184baf60ca83f69431b30df8a5e97a08478a9f2f9\": container with ID starting with 8e655781785e281ac275078184baf60ca83f69431b30df8a5e97a08478a9f2f9 not found: ID does not exist" containerID="8e655781785e281ac275078184baf60ca83f69431b30df8a5e97a08478a9f2f9"
Jul 20 17:32:12.328629 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.328581    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e655781785e281ac275078184baf60ca83f69431b30df8a5e97a08478a9f2f9"} err="failed to get container status \"8e655781785e281ac275078184baf60ca83f69431b30df8a5e97a08478a9f2f9\": rpc error: code = NotFound desc = could not find container \"8e655781785e281ac275078184baf60ca83f69431b30df8a5e97a08478a9f2f9\": container with ID starting with 8e655781785e281ac275078184baf60ca83f69431b30df8a5e97a08478a9f2f9 not found: ID does not exist"
Jul 20 17:32:12.341892 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.341868    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"]
Jul 20 17:32:12.347434 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.347364    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f5995d/rate-test-model-f5995d-kserve-54f549c79f-5zl97"]
Jul 20 17:32:12.896844 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:12.896810    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d57bf203-9e00-41b3-8956-7ad7239c2981" path="/var/lib/kubelet/pods/d57bf203-9e00-41b3-8956-7ad7239c2981/volumes"
Jul 20 17:32:17.290497 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:17.290461    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:17.391729 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:17.391693    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:27.426173 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:27.426128    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"]
Jul 20 17:32:27.426732 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:27.426409    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9" podUID="2bd62e71-5961-4339-a844-fd8e15b31685" containerName="maas-api" containerID="cri-o://53cc1eb86c2c21d36f4ac9941a53d1eb67f7371a0020c9dc54b7d5a523357e0c" gracePeriod=30
Jul 20 17:32:27.463291 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:27.463263    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:27.677417 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:27.677354    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:32:27.848233 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:27.848189    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd62e71-5961-4339-a844-fd8e15b31685-openshift-service-ca\") pod \"2bd62e71-5961-4339-a844-fd8e15b31685\" (UID: \"2bd62e71-5961-4339-a844-fd8e15b31685\") "
Jul 20 17:32:27.848421 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:27.848282    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9wzw\" (UniqueName: \"kubernetes.io/projected/2bd62e71-5961-4339-a844-fd8e15b31685-kube-api-access-v9wzw\") pod \"2bd62e71-5961-4339-a844-fd8e15b31685\" (UID: \"2bd62e71-5961-4339-a844-fd8e15b31685\") "
Jul 20 17:32:27.848421 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:27.848312    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd62e71-5961-4339-a844-fd8e15b31685-maas-api-tls\") pod \"2bd62e71-5961-4339-a844-fd8e15b31685\" (UID: \"2bd62e71-5961-4339-a844-fd8e15b31685\") "
Jul 20 17:32:27.848637 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:27.848613    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bd62e71-5961-4339-a844-fd8e15b31685-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2bd62e71-5961-4339-a844-fd8e15b31685" (UID: "2bd62e71-5961-4339-a844-fd8e15b31685"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:32:27.850575 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:27.850542    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bd62e71-5961-4339-a844-fd8e15b31685-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2bd62e71-5961-4339-a844-fd8e15b31685" (UID: "2bd62e71-5961-4339-a844-fd8e15b31685"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:32:27.850701 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:27.850581    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bd62e71-5961-4339-a844-fd8e15b31685-kube-api-access-v9wzw" (OuterVolumeSpecName: "kube-api-access-v9wzw") pod "2bd62e71-5961-4339-a844-fd8e15b31685" (UID: "2bd62e71-5961-4339-a844-fd8e15b31685"). InnerVolumeSpecName "kube-api-access-v9wzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:32:27.949847 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:27.949762    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd62e71-5961-4339-a844-fd8e15b31685-openshift-service-ca\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:32:27.949847 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:27.949789    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9wzw\" (UniqueName: \"kubernetes.io/projected/2bd62e71-5961-4339-a844-fd8e15b31685-kube-api-access-v9wzw\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:32:27.949847 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:27.949800    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd62e71-5961-4339-a844-fd8e15b31685-maas-api-tls\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:32:28.403517 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:28.403478    2568 generic.go:356] "Generic (PLEG): container finished" podID="2bd62e71-5961-4339-a844-fd8e15b31685" containerID="53cc1eb86c2c21d36f4ac9941a53d1eb67f7371a0020c9dc54b7d5a523357e0c" exitCode=0
Jul 20 17:32:28.403702 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:28.403528    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"
Jul 20 17:32:28.403702 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:28.403534    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9" event={"ID":"2bd62e71-5961-4339-a844-fd8e15b31685","Type":"ContainerDied","Data":"53cc1eb86c2c21d36f4ac9941a53d1eb67f7371a0020c9dc54b7d5a523357e0c"}
Jul 20 17:32:28.403702 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:28.403587    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9" event={"ID":"2bd62e71-5961-4339-a844-fd8e15b31685","Type":"ContainerDied","Data":"a46dd4d4188c3d5280d0f566d72859f367c54c4bd4a2baae60e62326945d4915"}
Jul 20 17:32:28.403702 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:28.403611    2568 scope.go:117] "RemoveContainer" containerID="53cc1eb86c2c21d36f4ac9941a53d1eb67f7371a0020c9dc54b7d5a523357e0c"
Jul 20 17:32:28.412853 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:28.412834    2568 scope.go:117] "RemoveContainer" containerID="53cc1eb86c2c21d36f4ac9941a53d1eb67f7371a0020c9dc54b7d5a523357e0c"
Jul 20 17:32:28.413103 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:32:28.413085    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53cc1eb86c2c21d36f4ac9941a53d1eb67f7371a0020c9dc54b7d5a523357e0c\": container with ID starting with 53cc1eb86c2c21d36f4ac9941a53d1eb67f7371a0020c9dc54b7d5a523357e0c not found: ID does not exist" containerID="53cc1eb86c2c21d36f4ac9941a53d1eb67f7371a0020c9dc54b7d5a523357e0c"
Jul 20 17:32:28.413159 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:28.413112    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53cc1eb86c2c21d36f4ac9941a53d1eb67f7371a0020c9dc54b7d5a523357e0c"} err="failed to get container status \"53cc1eb86c2c21d36f4ac9941a53d1eb67f7371a0020c9dc54b7d5a523357e0c\": rpc error: code = NotFound desc = could not find container \"53cc1eb86c2c21d36f4ac9941a53d1eb67f7371a0020c9dc54b7d5a523357e0c\": container with ID starting with 53cc1eb86c2c21d36f4ac9941a53d1eb67f7371a0020c9dc54b7d5a523357e0c not found: ID does not exist"
Jul 20 17:32:28.425456 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:28.425430    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"]
Jul 20 17:32:28.428808 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:28.428785    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f5995d-75564999c-lnlj9"]
Jul 20 17:32:28.896123 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:28.896085    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bd62e71-5961-4339-a844-fd8e15b31685" path="/var/lib/kubelet/pods/2bd62e71-5961-4339-a844-fd8e15b31685/volumes"
Jul 20 17:32:31.838947 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:31.838910    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:32.271582 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:32.271552    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:38.212536 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:38.212501    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:38.312758 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:38.312723    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:38.499889 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:38.499783    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:38.728187 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:38.728140    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:39.484869 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:39.484836    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:39.587597 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:39.587562    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:43.064676 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:43.064638    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:49.450583 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:49.450550    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"]
Jul 20 17:32:49.451082 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:49.450746    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg" podUID="55ecce5e-8715-4762-bcf5-f62264b70a64" containerName="maas-api" containerID="cri-o://749cea6988c74859332ad3dfe031d80f1e699f6cc624a5f99145216c27ea4758" gracePeriod=30
Jul 20 17:32:49.483949 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:49.483915    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:49.904493 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:49.904470    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:32:49.938028 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:49.937999    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ecce5e-8715-4762-bcf5-f62264b70a64-maas-api-tls\") pod \"55ecce5e-8715-4762-bcf5-f62264b70a64\" (UID: \"55ecce5e-8715-4762-bcf5-f62264b70a64\") "
Jul 20 17:32:49.938165 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:49.938082    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ecce5e-8715-4762-bcf5-f62264b70a64-openshift-service-ca\") pod \"55ecce5e-8715-4762-bcf5-f62264b70a64\" (UID: \"55ecce5e-8715-4762-bcf5-f62264b70a64\") "
Jul 20 17:32:49.938165 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:49.938113    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmftx\" (UniqueName: \"kubernetes.io/projected/55ecce5e-8715-4762-bcf5-f62264b70a64-kube-api-access-wmftx\") pod \"55ecce5e-8715-4762-bcf5-f62264b70a64\" (UID: \"55ecce5e-8715-4762-bcf5-f62264b70a64\") "
Jul 20 17:32:49.938471 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:49.938448    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ecce5e-8715-4762-bcf5-f62264b70a64-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55ecce5e-8715-4762-bcf5-f62264b70a64" (UID: "55ecce5e-8715-4762-bcf5-f62264b70a64"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:32:49.940383 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:49.940356    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ecce5e-8715-4762-bcf5-f62264b70a64-kube-api-access-wmftx" (OuterVolumeSpecName: "kube-api-access-wmftx") pod "55ecce5e-8715-4762-bcf5-f62264b70a64" (UID: "55ecce5e-8715-4762-bcf5-f62264b70a64"). InnerVolumeSpecName "kube-api-access-wmftx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:32:49.940538 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:49.940401    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ecce5e-8715-4762-bcf5-f62264b70a64-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55ecce5e-8715-4762-bcf5-f62264b70a64" (UID: "55ecce5e-8715-4762-bcf5-f62264b70a64"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:32:50.039757 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:50.039665    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ecce5e-8715-4762-bcf5-f62264b70a64-maas-api-tls\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:32:50.039757 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:50.039699    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ecce5e-8715-4762-bcf5-f62264b70a64-openshift-service-ca\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:32:50.039757 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:50.039708    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmftx\" (UniqueName: \"kubernetes.io/projected/55ecce5e-8715-4762-bcf5-f62264b70a64-kube-api-access-wmftx\") on node \"ip-10-0-128-8.ec2.internal\" DevicePath \"\""
Jul 20 17:32:50.512204 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:50.512168    2568 generic.go:356] "Generic (PLEG): container finished" podID="55ecce5e-8715-4762-bcf5-f62264b70a64" containerID="749cea6988c74859332ad3dfe031d80f1e699f6cc624a5f99145216c27ea4758" exitCode=0
Jul 20 17:32:50.512698 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:50.512221    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"
Jul 20 17:32:50.512698 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:50.512230    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg" event={"ID":"55ecce5e-8715-4762-bcf5-f62264b70a64","Type":"ContainerDied","Data":"749cea6988c74859332ad3dfe031d80f1e699f6cc624a5f99145216c27ea4758"}
Jul 20 17:32:50.512698 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:50.512317    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg" event={"ID":"55ecce5e-8715-4762-bcf5-f62264b70a64","Type":"ContainerDied","Data":"2dbdfc3e3be6f8d898080d216ff62e4ad832340b1949d2d0fe6f5cd0bab8259c"}
Jul 20 17:32:50.512698 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:50.512341    2568 scope.go:117] "RemoveContainer" containerID="749cea6988c74859332ad3dfe031d80f1e699f6cc624a5f99145216c27ea4758"
Jul 20 17:32:50.521435 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:50.521418    2568 scope.go:117] "RemoveContainer" containerID="749cea6988c74859332ad3dfe031d80f1e699f6cc624a5f99145216c27ea4758"
Jul 20 17:32:50.521705 ip-10-0-128-8 kubenswrapper[2568]: E0720 17:32:50.521685    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"749cea6988c74859332ad3dfe031d80f1e699f6cc624a5f99145216c27ea4758\": container with ID starting with 749cea6988c74859332ad3dfe031d80f1e699f6cc624a5f99145216c27ea4758 not found: ID does not exist" containerID="749cea6988c74859332ad3dfe031d80f1e699f6cc624a5f99145216c27ea4758"
Jul 20 17:32:50.521752 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:50.521714    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"749cea6988c74859332ad3dfe031d80f1e699f6cc624a5f99145216c27ea4758"} err="failed to get container status \"749cea6988c74859332ad3dfe031d80f1e699f6cc624a5f99145216c27ea4758\": rpc error: code = NotFound desc = could not find container \"749cea6988c74859332ad3dfe031d80f1e699f6cc624a5f99145216c27ea4758\": container with ID starting with 749cea6988c74859332ad3dfe031d80f1e699f6cc624a5f99145216c27ea4758 not found: ID does not exist"
Jul 20 17:32:50.534798 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:50.534772    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"]
Jul 20 17:32:50.539701 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:50.539682    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a24b97-58dd966697-wl6lg"]
Jul 20 17:32:50.897544 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:50.897510    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55ecce5e-8715-4762-bcf5-f62264b70a64" path="/var/lib/kubelet/pods/55ecce5e-8715-4762-bcf5-f62264b70a64/volumes"
Jul 20 17:32:53.932894 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:53.932854    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:32:54.347421 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:32:54.347336    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:33:00.300316 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:00.300269    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:33:00.400222 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:00.400188    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:33:00.561166 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:00.561077    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:33:00.793941 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:00.793911    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:33:06.630533 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:06.630493    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lxdxp_e43ba35f-7425-48b4-a81c-49dcfb086017/manager/0.log"
Jul 20 17:33:06.753838 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:06.753809    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kd9c8_5acb4774-f9cf-4e43-94b1-d990d3033b11/manager/0.log"
Jul 20 17:33:06.877732 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:06.877702    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77b8bc97bf-bvk88_2cfb5b69-20eb-4e61-a1a3-0b7cc362a952/manager/0.log"
Jul 20 17:33:06.992264 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:06.992171    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zrjc7_134f64e8-eb77-4709-9af2-f2081bbf2aaa/server/0.log"
Jul 20 17:33:07.115574 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:07.115545    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7l9vf_9a98944f-4afb-4716-978a-58b04ff33c50/manager/2.log"
Jul 20 17:33:07.378340 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:07.378313    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76b8cd99f7-f9gv6_af5d97d7-26f3-49f6-b2b2-44ae102a5ab4/copy-manifests/0.log"
Jul 20 17:33:07.385087 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:07.385063    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76b8cd99f7-f9gv6_af5d97d7-26f3-49f6-b2b2-44ae102a5ab4/manager/0.log"
Jul 20 17:33:07.878277 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:07.878232    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5df756f6bf-tfpb6_2cde84b1-5fcf-47d5-8d7e-19a872432489/maas-api/0.log"
Jul 20 17:33:09.771107 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:09.771062    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dksk2_24cfeb48-8cde-4a53-9d68-0eb5c6973ab7/manager/0.log"
Jul 20 17:33:09.883924 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:09.883896    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hfghk_9351b069-61d0-490c-a90b-a8b68fa167dd/manager/2.log"
Jul 20 17:33:10.004089 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:10.004059    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-4g64s_2929cae6-b7cf-4740-b628-635c1caf86d0/kuadrant-console-plugin/0.log"
Jul 20 17:33:10.127102 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:10.127071    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-q2flc_308a46b9-9916-430e-9532-be14539dbc3d/registry-server/0.log"
Jul 20 17:33:10.250790 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:10.250758    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-sqdq8_387dab05-928e-40ef-8944-8f58b98e4f4c/manager/0.log"
Jul 20 17:33:10.486758 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:10.486676    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-f2n7m_954a9eb2-1593-4471-abf5-f30e49776c4c/manager/2.log"
Jul 20 17:33:11.076445 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:11.076412    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-769f9dd78c-9gdz7_b6e7bb43-215a-4c22-9621-b4ca25e1c223/kube-auth-proxy/0.log"
Jul 20 17:33:11.447477 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:11.447443    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-9kjtn_dc696b70-0020-4372-baf8-0457fefc2593/payload-pre-processing/0.log"
Jul 20 17:33:11.571310 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:11.571278    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-ffpqs_3e638f7d-358e-444f-87d4-dc7c40e95588/payload-processing/0.log"
Jul 20 17:33:12.043139 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:12.043108    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-krp4f_a89f74e2-6a6a-4dec-a25d-50a41f3b0f33/main/0.log"
Jul 20 17:33:12.271206 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:12.271170    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-pjrfg_6e66bf16-461e-4af6-995d-7fbff1dd5cf6/main/0.log"
Jul 20 17:33:12.607882 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:12.607852    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-8fzxh_dea395ba-58f4-4e60-91b6-c85002ff099f/main/0.log"
Jul 20 17:33:17.969083 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:17.969045    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:33:18.069618 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:18.069583    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:33:20.921934 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:20.921906    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ntr6q_e1e4a352-901a-4050-b0c4-5e63ec823420/global-pull-secret-syncer/0.log"
Jul 20 17:33:21.028254 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:21.028218    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log"
Jul 20 17:33:21.118996 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:21.118965    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-8.ec2.internal_e1dddf4097058c612737a8bf9968fc66/haproxy/0.log"
Jul 20 17:33:26.504317 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:26.504282    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dksk2_24cfeb48-8cde-4a53-9d68-0eb5c6973ab7/manager/0.log"
Jul 20 17:33:26.530176 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:26.530146    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hfghk_9351b069-61d0-490c-a90b-a8b68fa167dd/manager/1.log"
Jul 20 17:33:26.530828 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:26.530806    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hfghk_9351b069-61d0-490c-a90b-a8b68fa167dd/manager/2.log"
Jul 20 17:33:26.549441 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:26.549421    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-4g64s_2929cae6-b7cf-4740-b628-635c1caf86d0/kuadrant-console-plugin/0.log"
Jul 20 17:33:26.602686 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:26.602660    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-q2flc_308a46b9-9916-430e-9532-be14539dbc3d/registry-server/0.log"
Jul 20 17:33:26.765315 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:26.765196    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-sqdq8_387dab05-928e-40ef-8944-8f58b98e4f4c/manager/0.log"
Jul 20 17:33:26.810886 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:26.810859    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-f2n7m_954a9eb2-1593-4471-abf5-f30e49776c4c/manager/1.log"
Jul 20 17:33:26.841560 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:26.841537    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-f2n7m_954a9eb2-1593-4471-abf5-f30e49776c4c/manager/2.log"
Jul 20 17:33:29.102127 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:29.102097    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q7946_231208e4-192f-4ca2-bd06-78c636016542/node-exporter/0.log"
Jul 20 17:33:29.123303 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:29.123232    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q7946_231208e4-192f-4ca2-bd06-78c636016542/kube-rbac-proxy/0.log"
Jul 20 17:33:29.144100 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:29.144082    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q7946_231208e4-192f-4ca2-bd06-78c636016542/init-textfile/0.log"
Jul 20 17:33:30.869011 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:30.868988    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-f4bbh_c63ca7b6-b603-474b-bc8a-66d0d0fba0f9/networking-console-plugin/0.log"
Jul 20 17:33:31.541298 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.541266    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"]
Jul 20 17:33:31.541640 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.541628    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ecce5e-8715-4762-bcf5-f62264b70a64" containerName="maas-api"
Jul 20 17:33:31.541684 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.541642    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ecce5e-8715-4762-bcf5-f62264b70a64" containerName="maas-api"
Jul 20 17:33:31.541684 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.541653    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bd62e71-5961-4339-a844-fd8e15b31685" containerName="maas-api"
Jul 20 17:33:31.541684 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.541658    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bd62e71-5961-4339-a844-fd8e15b31685" containerName="maas-api"
Jul 20 17:33:31.541684 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.541668    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d57bf203-9e00-41b3-8956-7ad7239c2981" containerName="main"
Jul 20 17:33:31.541684 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.541674    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d57bf203-9e00-41b3-8956-7ad7239c2981" containerName="main"
Jul 20 17:33:31.541853 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.541688    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8ac2f41-b499-4134-a650-0d6158c04098" containerName="main"
Jul 20 17:33:31.541853 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.541694    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8ac2f41-b499-4134-a650-0d6158c04098" containerName="main"
Jul 20 17:33:31.541853 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.541751    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8ac2f41-b499-4134-a650-0d6158c04098" containerName="main"
Jul 20 17:33:31.541853 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.541759    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bd62e71-5961-4339-a844-fd8e15b31685" containerName="maas-api"
Jul 20 17:33:31.541853 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.541765    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ecce5e-8715-4762-bcf5-f62264b70a64" containerName="maas-api"
Jul 20 17:33:31.541853 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.541770    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d57bf203-9e00-41b3-8956-7ad7239c2981" containerName="main"
Jul 20 17:33:31.549774 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.549737    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.552413 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.552383    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"]
Jul 20 17:33:31.552848 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.552820    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z68mp\"/\"openshift-service-ca.crt\""
Jul 20 17:33:31.553928 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.553905    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z68mp\"/\"default-dockercfg-58xlt\""
Jul 20 17:33:31.554036 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.553933    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z68mp\"/\"kube-root-ca.crt\""
Jul 20 17:33:31.595294 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.595267    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b5f0748f-c627-4daf-b400-04f467205d87-podres\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.595457 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.595300    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npjk5\" (UniqueName: \"kubernetes.io/projected/b5f0748f-c627-4daf-b400-04f467205d87-kube-api-access-npjk5\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.595457 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.595345    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5f0748f-c627-4daf-b400-04f467205d87-lib-modules\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.595457 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.595374    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b5f0748f-c627-4daf-b400-04f467205d87-proc\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.595597 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.595480    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5f0748f-c627-4daf-b400-04f467205d87-sys\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.696884 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.696843    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5f0748f-c627-4daf-b400-04f467205d87-sys\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.697072 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.696916    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b5f0748f-c627-4daf-b400-04f467205d87-podres\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.697072 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.696941    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npjk5\" (UniqueName: \"kubernetes.io/projected/b5f0748f-c627-4daf-b400-04f467205d87-kube-api-access-npjk5\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.697072 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.696980    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5f0748f-c627-4daf-b400-04f467205d87-sys\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.697072 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.697025    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5f0748f-c627-4daf-b400-04f467205d87-lib-modules\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.697072 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.697062    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b5f0748f-c627-4daf-b400-04f467205d87-proc\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.697377 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.697084    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b5f0748f-c627-4daf-b400-04f467205d87-podres\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.697377 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.697150    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b5f0748f-c627-4daf-b400-04f467205d87-proc\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.697377 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.697210    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5f0748f-c627-4daf-b400-04f467205d87-lib-modules\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.705569 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.705543    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npjk5\" (UniqueName: \"kubernetes.io/projected/b5f0748f-c627-4daf-b400-04f467205d87-kube-api-access-npjk5\") pod \"perf-node-gather-daemonset-z66tz\" (UID: \"b5f0748f-c627-4daf-b400-04f467205d87\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.861290 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.861176    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:31.997210 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:31.997185    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"]
Jul 20 17:33:31.999465 ip-10-0-128-8 kubenswrapper[2568]: W0720 17:33:31.999436    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb5f0748f_c627_4daf_b400_04f467205d87.slice/crio-0e95de68ed981875f75dd4958b3f10176ee2556087c9145f348aaf872d5ae657 WatchSource:0}: Error finding container 0e95de68ed981875f75dd4958b3f10176ee2556087c9145f348aaf872d5ae657: Status 404 returned error can't find the container with id 0e95de68ed981875f75dd4958b3f10176ee2556087c9145f348aaf872d5ae657
Jul 20 17:33:32.723716 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:32.723674    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz" event={"ID":"b5f0748f-c627-4daf-b400-04f467205d87","Type":"ContainerStarted","Data":"95ffd17d757faafe9aeb19fb6398d2c5c799cbe8ed2fddd68363ac06b3c4f846"}
Jul 20 17:33:32.723939 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:32.723734    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz" event={"ID":"b5f0748f-c627-4daf-b400-04f467205d87","Type":"ContainerStarted","Data":"0e95de68ed981875f75dd4958b3f10176ee2556087c9145f348aaf872d5ae657"}
Jul 20 17:33:32.723939 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:32.723792    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:32.752788 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:32.752747    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz" podStartSLOduration=1.752735438 podStartE2EDuration="1.752735438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:33:32.751692056 +0000 UTC m=+2892.299548122" watchObservedRunningTime="2026-07-20 17:33:32.752735438 +0000 UTC m=+2892.300591505"
Jul 20 17:33:33.911160 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:33.911119    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m4g87_372d96f7-664c-4583-a4f2-5f4af55157a1/dns/0.log"
Jul 20 17:33:33.930447 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:33.930425    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m4g87_372d96f7-664c-4583-a4f2-5f4af55157a1/kube-rbac-proxy/0.log"
Jul 20 17:33:33.959748 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:33.959719    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6g5w2_798679f7-4546-4546-bc3d-c12693574272/dns-node-resolver/0.log"
Jul 20 17:33:34.567720 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:34.567695    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6f6f846bcf-jgw5v_7c9724d1-ce8b-46de-b7fb-851f8aba5b98/registry/0.log"
Jul 20 17:33:34.640054 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:34.640018    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sfzwq_46c16140-8350-4ab4-9e33-b92a7fb16f1e/node-ca/0.log"
Jul 20 17:33:35.739556 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:35.739528    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-769f9dd78c-9gdz7_b6e7bb43-215a-4c22-9621-b4ca25e1c223/kube-auth-proxy/0.log"
Jul 20 17:33:35.858470 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:35.858439    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-9kjtn_dc696b70-0020-4372-baf8-0457fefc2593/payload-pre-processing/0.log"
Jul 20 17:33:35.896375 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:35.896341    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-ffpqs_3e638f7d-358e-444f-87d4-dc7c40e95588/payload-processing/0.log"
Jul 20 17:33:36.675153 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:36.675115    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p2vpf_d5e37e15-6c9c-4b17-b026-5ae24cbe3e04/serve-healthcheck-canary/0.log"
Jul 20 17:33:37.174127 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:37.174084    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d8b4g_30638c7b-0d95-4b98-947d-829c24bb32de/kube-rbac-proxy/0.log"
Jul 20 17:33:37.197464 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:37.197439    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d8b4g_30638c7b-0d95-4b98-947d-829c24bb32de/exporter/0.log"
Jul 20 17:33:37.222007 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:37.221984    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d8b4g_30638c7b-0d95-4b98-947d-829c24bb32de/extractor/0.log"
Jul 20 17:33:38.739369 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:38.739339    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-z66tz"
Jul 20 17:33:38.999768 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:38.999685    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8phrk_9b3bec34-0628-43fe-9211-2203141bd83c/konnectivity-agent/0.log" limit=1024
Jul 20 17:33:39.592737 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:39.592713    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lxdxp_e43ba35f-7425-48b4-a81c-49dcfb086017/manager/0.log"
Jul 20 17:33:39.912610 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:39.912575    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kd9c8_5acb4774-f9cf-4e43-94b1-d990d3033b11/manager/0.log"
Jul 20 17:33:40.109703 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:40.109668    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77b8bc97bf-bvk88_2cfb5b69-20eb-4e61-a1a3-0b7cc362a952/manager/0.log"
Jul 20 17:33:40.134288 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:40.134257    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zrjc7_134f64e8-eb77-4709-9af2-f2081bbf2aaa/server/0.log"
Jul 20 17:33:40.160978 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:40.160949    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7l9vf_9a98944f-4afb-4716-978a-58b04ff33c50/manager/1.log"
Jul 20 17:33:40.205935 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:40.205866    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7l9vf_9a98944f-4afb-4716-978a-58b04ff33c50/manager/2.log"
Jul 20 17:33:40.277365 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:40.277335    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76b8cd99f7-f9gv6_af5d97d7-26f3-49f6-b2b2-44ae102a5ab4/manager/0.log"
Jul 20 17:33:40.296909 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:40.296879    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76b8cd99f7-f9gv6_af5d97d7-26f3-49f6-b2b2-44ae102a5ab4/copy-manifests/0.log"
Jul 20 17:33:42.253090 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:42.253064    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-769f9dd78c-9gdz7_b6e7bb43-215a-4c22-9621-b4ca25e1c223/kube-auth-proxy/0.log"
Jul 20 17:33:42.349545 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:42.349492    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-9kjtn_dc696b70-0020-4372-baf8-0457fefc2593/payload-pre-processing/0.log"
Jul 20 17:33:42.377570 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:42.377542    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-ffpqs_3e638f7d-358e-444f-87d4-dc7c40e95588/payload-processing/0.log"
Jul 20 17:33:43.800175 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:43.800148    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-769f9dd78c-9gdz7_b6e7bb43-215a-4c22-9621-b4ca25e1c223/kube-auth-proxy/0.log"
Jul 20 17:33:43.898788 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:43.898754    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-9kjtn_dc696b70-0020-4372-baf8-0457fefc2593/payload-pre-processing/0.log"
Jul 20 17:33:43.928856 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:43.928816    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-ffpqs_3e638f7d-358e-444f-87d4-dc7c40e95588/payload-processing/0.log"
Jul 20 17:33:47.006877 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:47.006836    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-tclrk_2106a93a-fea7-4adb-9933-46cb91da712d/migrator/0.log"
Jul 20 17:33:47.034486 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:47.034458    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-tclrk_2106a93a-fea7-4adb-9933-46cb91da712d/graceful-termination/0.log"
Jul 20 17:33:48.846695 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:48.846663    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f844c_ff628bb3-d193-412f-bae5-c23349f488da/kube-multus-additional-cni-plugins/0.log"
Jul 20 17:33:48.867296 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:48.867230    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f844c_ff628bb3-d193-412f-bae5-c23349f488da/egress-router-binary-copy/0.log"
Jul 20 17:33:48.890939 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:48.890872    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f844c_ff628bb3-d193-412f-bae5-c23349f488da/cni-plugins/0.log"
Jul 20 17:33:48.911858 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:48.911831    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f844c_ff628bb3-d193-412f-bae5-c23349f488da/bond-cni-plugin/0.log"
Jul 20 17:33:48.933774 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:48.933746    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f844c_ff628bb3-d193-412f-bae5-c23349f488da/routeoverride-cni/0.log"
Jul 20 17:33:48.955803 ip-10-0-128-8 kubenswrapper[2568]: I0720 17:33:48.955770    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f844c_ff628bb3-d193-412f-bae5-c23349f488da/whereabouts-cni-bincopy/0.log"