Jul 20 22:21:00.866621 ip-10-0-129-16 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 20 22:21:00.866632 ip-10-0-129-16 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 20 22:21:00.866642 ip-10-0-129-16 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 20 22:21:00.866974 ip-10-0-129-16 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 20 22:21:11.055464 ip-10-0-129-16 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 20 22:21:11.055481 ip-10-0-129-16 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 3103dca8b577484388f2f15b895d7f07 --
Jul 20 22:22:42.905584 ip-10-0-129-16 systemd[1]: Starting Kubernetes Kubelet...
Jul 20 22:22:43.286214 ip-10-0-129-16 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 22:22:43.286214 ip-10-0-129-16 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 20 22:22:43.286214 ip-10-0-129-16 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 22:22:43.286214 ip-10-0-129-16 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 20 22:22:43.286214 ip-10-0-129-16 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 22:22:43.287254 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.287172    2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 20 22:22:43.291782 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291768    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 22:22:43.291782 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291782    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291787    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291790    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291793    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291795    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291798    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291801    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291804    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291807    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291809    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291812    2550 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291814    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291817    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291820    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291822    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291825    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291827    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291830    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291837    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291840    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 22:22:43.291845 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291843    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291845    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291848    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291850    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291853    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291856    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291859    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291862    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291864    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291867    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291869    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291872    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291875    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291877    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291879    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291882    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291884    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291887    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291889    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291892    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 22:22:43.292327 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291895    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291897    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291899    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291904    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291908    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291912    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291915    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291918    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291921    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291924    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291927    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291930    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291932    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291935    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291938    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291941    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291944    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291946    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291949    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 22:22:43.292844 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291951    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291954    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291956    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291959    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291962    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291964    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291966    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291970    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291972    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291975    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291978    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291980    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291983    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291987    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291991    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291994    2550 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291996    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.291999    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292001    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 22:22:43.293308 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292003    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292006    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292009    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292011    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292014    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292016    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292019    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292022    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292025    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292028    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292030    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292033    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292036    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292038    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292041    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292043    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292045    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292048    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292050    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292053    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 22:22:43.293765 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292783    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292790    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292794    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292797    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292800    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292802    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292805    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292808    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292810    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292813    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292816    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292820    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292824    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292826    2550 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292829    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292832    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292834    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292837    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292839    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292842    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 22:22:43.294267 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292847    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292850    2550 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292852    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292855    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292858    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292861    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292864    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292866    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292869    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292871    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292874    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292876    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292879    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292882    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292884    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292886    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292889    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292891    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292894    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292898    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 22:22:43.294805 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292901    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292904    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292907    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292910    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292913    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292915    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292918    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292920    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292923    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292925    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292929    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292933    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292937    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292940    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292943    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292945    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292948    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292950    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292953    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 22:22:43.295286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292955    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292958    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292960    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292962    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292965    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292967    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292970    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292972    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292975    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292978    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292980    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292983    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292986    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292988    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292991    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292993    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292996    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.292998    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293001    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293003    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 22:22:43.295761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293007    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293009    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293012    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293014    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293017    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293019    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293022    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293025    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293028    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293030    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293032    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293035    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293037    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293040    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293042    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293045    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293047    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293050    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293052    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293054    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 22:22:43.296232 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293483    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293488    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293491    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293494    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293497    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293499    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293502    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293505    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293508    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293511    2550 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293513    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293516    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293518    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293521    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293525    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293529    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293533    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293536    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293539    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293543    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 22:22:43.296721 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293546    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293549    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293551    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293554    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293557    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293559    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293562    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293565    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293567    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293569    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293572    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293574    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293577    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293579    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293582    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293585    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293588    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293591    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293593    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293596    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 22:22:43.297202 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293598    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293601    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293603    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293606    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293620    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293624    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293626    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293629    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293632    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293634    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293637    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293641    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293643    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293647    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293650    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293652    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293655    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293657    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293660    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 22:22:43.297695 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293662    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293665    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293667    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293670    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293672    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293675    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293678    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293683    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293686    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293688    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293691    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293693    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293696    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293698    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293701    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293703    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293706    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293711    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293714    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293717    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 22:22:43.298158 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293720    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293722    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293726    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293729    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293731    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293734    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293737    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293741    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293744    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293747    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293750    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293753    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293756    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293758    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293761    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293764    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293767    2550 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293770    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293772    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.293777    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 22:22:43.298648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293847    2550 flags.go:64] FLAG: --address="0.0.0.0"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293858    2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293864    2550 flags.go:64] FLAG: --anonymous-auth="true"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293869    2550 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293873    2550 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293876    2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293880    2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293884    2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293887    2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293890    2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293894    2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293897    2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293900    2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293903    2550 flags.go:64] FLAG: --cgroup-root=""
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293907    2550 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293910    2550 flags.go:64] FLAG: --client-ca-file=""
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293913    2550 flags.go:64] FLAG: --cloud-provider="external"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293916    2550 flags.go:64] FLAG: --cluster-dns="[]"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293921    2550 flags.go:64] FLAG: --cluster-domain=""
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293924    2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293927    2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293930    2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293933    2550 flags.go:64] FLAG: --container-log-max-files="5"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293937    2550 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 20 22:22:43.299166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293941    2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293944    2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293947    2550 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293950    2550 flags.go:64] FLAG: --contention-profiling="false"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293953    2550 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293956    2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293959    2550 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293964    2550 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293967    2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293970    2550 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293973    2550 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293976    2550 flags.go:64] FLAG: --enable-load-reader="false"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293979    2550 flags.go:64] FLAG: --enable-server="true"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293982    2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293986    2550 flags.go:64] FLAG: --event-burst="100"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293990    2550 flags.go:64] FLAG: --event-qps="50"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293992    2550 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293995    2550 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.293998    2550 flags.go:64] FLAG: --eviction-hard=""
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294002    2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294005    2550 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294008    2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294012    2550 flags.go:64] FLAG: --eviction-soft=""
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294015    2550 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294018    2550 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 20 22:22:43.299736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294021    2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294024    2550 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294027    2550 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294030    2550 flags.go:64] FLAG: --fail-swap-on="true"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294033    2550 flags.go:64] FLAG: --feature-gates=""
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294036    2550 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294039    2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294042    2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294045    2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294048    2550 flags.go:64] FLAG: --healthz-port="10248"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294051    2550 flags.go:64] FLAG: --help="false"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294055    2550 flags.go:64] FLAG: --hostname-override="ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294058    2550 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294061    2550 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294065    2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294068    2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294072    2550 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294075    2550 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294077    2550 flags.go:64] FLAG: --image-service-endpoint=""
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294080    2550 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294083    2550 flags.go:64] FLAG: --kube-api-burst="100"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294085    2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294089    2550 flags.go:64] FLAG: --kube-api-qps="50"
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294091    2550 flags.go:64] FLAG: --kube-reserved=""
Jul 20 22:22:43.300381 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294094    2550 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294097    2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294100    2550 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294103    2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294106    2550 flags.go:64] FLAG: --lock-file=""
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294110    2550 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294113    2550 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294118    2550 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294125    2550 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294128    2550 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294132    2550 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294135    2550 flags.go:64] FLAG: --logging-format="text"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294138    2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294142    2550 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294145    2550 flags.go:64] FLAG: --manifest-url=""
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294147    2550 flags.go:64] FLAG: --manifest-url-header=""
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294151    2550 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294155    2550 flags.go:64] FLAG: --max-open-files="1000000"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294159    2550 flags.go:64] FLAG: --max-pods="110"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294162    2550 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294165    2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294168    2550 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294172    2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294175    2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294178    2550 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 20 22:22:43.301367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294181    2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294188    2550 flags.go:64] FLAG: --node-status-max-images="50"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294190    2550 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294194    2550 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294197    2550 flags.go:64] FLAG: --pod-cidr=""
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294199    2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294205    2550 flags.go:64] FLAG: --pod-manifest-path=""
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294207    2550 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294210    2550 flags.go:64] FLAG: --pods-per-core="0"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294213    2550 flags.go:64] FLAG: --port="10250"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294216    2550 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294219    2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e6f0a2312e080d2d"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294223    2550 flags.go:64] FLAG: --qos-reserved=""
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294226    2550 flags.go:64] FLAG: --read-only-port="10255"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294229    2550 flags.go:64] FLAG: --register-node="true"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294232    2550 flags.go:64] FLAG: --register-with-taints=""
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294236    2550 flags.go:64] FLAG: --registry-burst="10"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294239    2550 flags.go:64] FLAG: --registry-qps="5"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294242    2550 flags.go:64] FLAG: --reserved-cpus=""
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294247    2550 flags.go:64] FLAG: --reserved-memory=""
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294251    2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294254    2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294257    2550 flags.go:64] FLAG: --rotate-certificates="false"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294260    2550 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294263    2550 flags.go:64] FLAG: --runonce="false"
Jul 20 22:22:43.302140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294266    2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294269    2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294272    2550 flags.go:64] FLAG: --seccomp-default="false"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294275    2550 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294278    2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294283    2550 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294286    2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294289    2550 flags.go:64] FLAG: --storage-driver-password="root"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294305    2550 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294308    2550 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294311    2550 flags.go:64] FLAG: --storage-driver-user="root"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294313    2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294317    2550 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294320    2550 flags.go:64] FLAG: --system-cgroups=""
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294322    2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294328    2550 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294331    2550 flags.go:64] FLAG: --tls-cert-file=""
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294334    2550 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294337    2550 flags.go:64] FLAG: --tls-min-version=""
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294341    2550 flags.go:64] FLAG: --tls-private-key-file=""
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294344    2550 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294347    2550 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294350    2550 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294353    2550 flags.go:64] FLAG: --v="2"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294357    2550 flags.go:64] FLAG: --version="false"
Jul 20 22:22:43.302855 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294362    2550 flags.go:64] FLAG: --vmodule=""
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294366    2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294369    2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294485    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294489    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294492    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294495    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294497    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294500    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294503    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294505    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294508    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294511    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294514    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294516    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294519    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294521    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294524    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294526    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294529    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 22:22:43.303568 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294532    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294534    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294537    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294540    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294543    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294545    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294549    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294552    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294554    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294557    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294559    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294562    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294565    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294567    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294569    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294572    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294576    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294580    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294582    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 22:22:43.304097 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294585    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294588    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294591    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294594    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294596    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294599    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294602    2550 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294604    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294607    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294609    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294612    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294614    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294617    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294620    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294622    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294625    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294627    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294629    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294632    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294636    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 22:22:43.304630 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294638    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294641    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294643    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294646    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294648    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294651    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294653    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294656    2550 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294658    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294661    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294664    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294666    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294669    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294671    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294675    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294679    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294683    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294686    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294689    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 22:22:43.305160 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294692    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294695    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294697    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294700    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294702    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294705    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294708    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294711    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294714    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294716    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294719    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294722    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294726    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294729    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294731    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294734    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294736    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294739    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294741    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294744    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 22:22:43.305761 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294746    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294749    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294751    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.294754    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.294760    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.303065    2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.303080    2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.303100    2550 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.303105    2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303162    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303167    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303171    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303174    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303177    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303180    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 22:22:43.306642 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303183    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303186    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303190    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303192    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303195    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303198    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303200    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303203    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303206    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303209    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303211    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303214    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303217    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303219    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303222    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303224    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303227    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303229    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303234    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303237    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 22:22:43.307105 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303240    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303243    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303246    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303249    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303252    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303255    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303260    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303263    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303266    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303269    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303272    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303275    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303278    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303282    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303284    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303287    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303289    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303307    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303310    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 22:22:43.307608 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303312    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303315    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303318    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303321    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303324    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303327    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303330    2550 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303333    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303336    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303339    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303341    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303344    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303347    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303349    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303352    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303355    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303358    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303360    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303363    2550 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303365    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 22:22:43.308059 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303369    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303371    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303374    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303376    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303379    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303381    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303384    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303387    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303389    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303392    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303394    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303397    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303399    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303402    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303405    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303407    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303410    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303412    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303415    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303418    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 22:22:43.308555 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303420    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303423    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303425    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303428    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303430    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303433    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303436    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303438    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303441    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303445    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303449    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303452    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303455    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303458    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 22:22:43.309026 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.303463    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303576    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303581    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303585    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303589    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303593    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303596    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303599    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303602    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303605    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303607    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303610    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303612    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303616    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303619    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303621    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303624    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303627    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303630    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 22:22:43.309390 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303632    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303635    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303638    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303640    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303643    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303645    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303648    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303650    2550 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303653    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303656    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303659    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303661    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303664    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303666    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303669    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303671    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303674    2550 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303676    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303679    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303681    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 22:22:43.309854 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303684    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303686    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303688    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303691    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303693    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303696    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303699    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303702    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303705    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303707    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303710    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303713    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303716    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303720    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303723    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303726    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303729    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303732    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303735    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 22:22:43.310410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303737    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303740    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303743    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303746    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303748    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303750    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303753    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303756    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303758    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303760    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303763    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303765    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303769    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303772    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303775    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303777    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303780    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303782    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303785    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 22:22:43.310861 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303787    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303790    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303798    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303801    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303804    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303806    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303809    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303811    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303814    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303816    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303818    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303821    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303823    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303826    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303828    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303831    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303833    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303836    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303839    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303841    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303843    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 22:22:43.311324 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303846    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 22:22:43.311820 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:43.303848    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 22:22:43.311820 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.303853    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 22:22:43.311820 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.303952    2550 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 20 22:22:43.456262 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.456230    2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 20 22:22:43.457730 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.457718    2550 server.go:1019] "Starting client certificate rotation"
Jul 20 22:22:43.457841 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.457820    2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 22:22:43.457871 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.457859    2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 22:22:43.460179 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.460166    2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 22:22:43.461594 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.461580    2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 22:22:43.471575 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.471554    2550 log.go:25] "Validated CRI v1 runtime API"
Jul 20 22:22:43.475366 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.475350    2550 log.go:25] "Validated CRI v1 image API"
Jul 20 22:22:43.476358 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.476343    2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 20 22:22:43.479920 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.479899    2550 fs.go:135] Filesystem UUIDs: map[146beccf-7bff-4c87-9081-9397d0d41199:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a6fe27fe-ab29-4b49-ae47-97e7b48e3a14:/dev/nvme0n1p4]
Jul 20 22:22:43.479968 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.479921    2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 20 22:22:43.485992 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.485872    2550 manager.go:216] Machine: {Timestamp:2026-07-20 22:22:43.484573815 +0000 UTC m=+0.468700692 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100297 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2315392d58c26e96d05e72dfd95934 SystemUUID:ec231539-2d58-c26e-96d0-5e72dfd95934 BootID:3103dca8-b577-4843-88f2-f15b895d7f07 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fa:c2:67:1b:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fa:c2:67:1b:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:07:50:b9:52:a2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 20 22:22:43.485992 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.485983    2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 20 22:22:43.486133 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.486068    2550 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 20 22:22:43.486720 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.486703    2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 22:22:43.489423 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.489391    2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 20 22:22:43.489558 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.489427    2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-16.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 20 22:22:43.490693 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.490683    2550 topology_manager.go:138] "Creating topology manager with none policy"
Jul 20 22:22:43.490733 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.490696    2550 container_manager_linux.go:309] "Creating device plugin manager"
Jul 20 22:22:43.490733 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.490704    2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 22:22:43.491337 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.491327    2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 22:22:43.492705 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.492695    2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 20 22:22:43.495687 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.495675    2550 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 22:22:43.495809 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.495799    2550 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 20 22:22:43.495850 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.495829    2550 kubelet.go:486] "Attempting to sync node with API server"
Jul 20 22:22:43.495850 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.495839    2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 20 22:22:43.495916 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.495854    2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 20 22:22:43.495916 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.495864    2550 kubelet.go:398] "Adding apiserver pod source"
Jul 20 22:22:43.495916 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.495872    2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 20 22:22:43.496206 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.496193    2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 22:22:43.496269 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.496213    2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 22:22:43.500875 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.500859    2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 20 22:22:43.502310 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.502282    2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 20 22:22:43.502385 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.502324    2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 20 22:22:43.502982 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.502969    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 20 22:22:43.502982 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.502985    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 20 22:22:43.503107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.502992    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 20 22:22:43.503107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.502997    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 20 22:22:43.503107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.503005    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 20 22:22:43.503107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.503011    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 20 22:22:43.503107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.503046    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 20 22:22:43.503107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.503052    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 20 22:22:43.503107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.503058    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 20 22:22:43.503107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.503064    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 20 22:22:43.503107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.503081    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 20 22:22:43.503107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.503090    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 20 22:22:43.504482 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.504471    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 20 22:22:43.504482 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.504481    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 20 22:22:43.506540 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.506524    2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-16.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 20 22:22:43.506627 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.506610    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 20 22:22:43.506692 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.506674    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 20 22:22:43.507555 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.507543    2550 server.go:1268] "Started kubelet"
Jul 20 22:22:43.507675 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.507638    2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 20 22:22:43.507784 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.507641    2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 20 22:22:43.507784 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.507747    2550 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 20 22:22:43.508396 ip-10-0-129-16 systemd[1]: Started Kubernetes Kubelet.
Jul 20 22:22:43.509051 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.509037    2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 20 22:22:43.510040 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.510027    2550 server.go:310] "Adding debug handlers to kubelet server"
Jul 20 22:22:43.513156 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.513135    2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 20 22:22:43.516803 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.516786    2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 20 22:22:43.516887 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.516787    2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 20 22:22:43.517031 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.517010    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:43.517756 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.517738    2550 factory.go:55] Registering systemd factory
Jul 20 22:22:43.517756 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.517758    2550 factory.go:223] Registration of the systemd container factory successfully
Jul 20 22:22:43.517956 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.517940    2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 20 22:22:43.518008 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.517987    2550 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 20 22:22:43.518008 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.518000    2550 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 20 22:22:43.518131 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.518116    2550 reconstruct.go:97] "Volume reconstruction finished"
Jul 20 22:22:43.518131 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.518132    2550 reconciler.go:29] "Reconciler: start to sync state"
Jul 20 22:22:43.518793 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.518775    2550 factory.go:153] Registering CRI-O factory
Jul 20 22:22:43.518793 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.518795    2550 factory.go:223] Registration of the crio container factory successfully
Jul 20 22:22:43.518916 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.518844    2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 20 22:22:43.518916 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.518870    2550 factory.go:103] Registering Raw factory
Jul 20 22:22:43.518916 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.518884    2550 manager.go:1196] Started watching for new ooms in manager
Jul 20 22:22:43.520533 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.520510    2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4qrx4"
Jul 20 22:22:43.520810 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.520790    2550 manager.go:349] Starting recovery of all containers
Jul 20 22:22:43.521998 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.521970    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 20 22:22:43.526773 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.526748    2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4qrx4"
Jul 20 22:22:43.526983 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.526957    2550 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 20 22:22:43.528089 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.527052    2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-16.ec2.internal.18c41fa91af15ce9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-16.ec2.internal,UID:ip-10-0-129-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-16.ec2.internal,},FirstTimestamp:2026-07-20 22:22:43.507518697 +0000 UTC m=+0.491645575,LastTimestamp:2026-07-20 22:22:43.507518697 +0000 UTC m=+0.491645575,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-16.ec2.internal,}"
Jul 20 22:22:43.531507 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.531492    2550 manager.go:354] Recovery completed
Jul 20 22:22:43.536137 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.536024    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 22:22:43.537953 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.537909    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 22:22:43.537953 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.537943    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 22:22:43.538059 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.537957    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Jul 20 22:22:43.538484 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.538468    2550 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 20 22:22:43.538484 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.538482    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 22:22:43.538603 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.538482    2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 20 22:22:43.538603 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.538546    2550 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 22:22:43.539150 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.539137    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 22:22:43.539196 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.539161    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 22:22:43.539196 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.539170    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Jul 20 22:22:43.539753 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.539669    2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-16.ec2.internal.18c41fa91cc15837  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-16.ec2.internal,UID:ip-10-0-129-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-16.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-16.ec2.internal,},FirstTimestamp:2026-07-20 22:22:43.537926199 +0000 UTC m=+0.522053076,LastTimestamp:2026-07-20 22:22:43.537926199 +0000 UTC m=+0.522053076,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-16.ec2.internal,}"
Jul 20 22:22:43.540003 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.539991    2550 policy_none.go:49] "None policy: Start"
Jul 20 22:22:43.540045 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.540007    2550 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 20 22:22:43.540045 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.540018    2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 20 22:22:43.541264 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.541253    2550 policy_none.go:47] "Start"
Jul 20 22:22:43.581647 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.581626    2550 manager.go:337] "Starting Device Plugin manager"
Jul 20 22:22:43.601179 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.581662    2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 20 22:22:43.601179 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.581678    2550 server.go:87] "Starting device plugin registration server"
Jul 20 22:22:43.601179 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.582182    2550 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 20 22:22:43.601179 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.582198    2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 20 22:22:43.601179 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.582307    2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 20 22:22:43.601179 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.582392    2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 20 22:22:43.601179 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.582401    2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 20 22:22:43.601179 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.582929    2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 20 22:22:43.601179 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.582984    2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:43.630417 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.630374    2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 20 22:22:43.631619 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.631601    2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 20 22:22:43.631674 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.631654    2550 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 20 22:22:43.631720 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.631681    2550 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 20 22:22:43.631757 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.631724    2550 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 20 22:22:43.633945 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.633929    2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 22:22:43.682918 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.682883    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 22:22:43.683911 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.683894    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 22:22:43.683980 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.683929    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 22:22:43.683980 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.683943    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Jul 20 22:22:43.683980 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.683964    2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.691897 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.691883    2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.691953 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.691905    2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-16.ec2.internal\": node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:43.709339 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.709316    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:43.732187 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.732159    2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"]
Jul 20 22:22:43.732260 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.732250    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 22:22:43.733194 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.733177    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 22:22:43.733289 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.733212    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 22:22:43.733289 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.733225    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Jul 20 22:22:43.734638 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.734624    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 22:22:43.734769 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.734751    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.734834 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.734781    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 22:22:43.735431 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.735413    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 22:22:43.735521 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.735419    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 22:22:43.735521 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.735462    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 22:22:43.735521 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.735472    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Jul 20 22:22:43.735521 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.735441    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 22:22:43.735716 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.735525    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Jul 20 22:22:43.736944 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.736927    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.737027 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.736969    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 22:22:43.739485 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.739467    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 22:22:43.739621 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.739496    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 22:22:43.739621 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.739511    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeHasSufficientPID"
Jul 20 22:22:43.754849 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.754827    2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-16.ec2.internal\" not found" node="ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.758283 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.758269    2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-16.ec2.internal\" not found" node="ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.810109 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.810025    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:43.819289 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.819260    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ba3a9b5625ee0077a9baad34e458c51-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal\" (UID: \"7ba3a9b5625ee0077a9baad34e458c51\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.819401 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.819307    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ba3a9b5625ee0077a9baad34e458c51-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal\" (UID: \"7ba3a9b5625ee0077a9baad34e458c51\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.819401 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.819325    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b8d2414fddfe8167462c801d0be4b0d-config\") pod \"kube-apiserver-proxy-ip-10-0-129-16.ec2.internal\" (UID: \"3b8d2414fddfe8167462c801d0be4b0d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.911015 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:43.910974    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:43.920453 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.920423    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b8d2414fddfe8167462c801d0be4b0d-config\") pod \"kube-apiserver-proxy-ip-10-0-129-16.ec2.internal\" (UID: \"3b8d2414fddfe8167462c801d0be4b0d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.920562 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.920473    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ba3a9b5625ee0077a9baad34e458c51-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal\" (UID: \"7ba3a9b5625ee0077a9baad34e458c51\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.920562 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.920498    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ba3a9b5625ee0077a9baad34e458c51-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal\" (UID: \"7ba3a9b5625ee0077a9baad34e458c51\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.920562 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.920435    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b8d2414fddfe8167462c801d0be4b0d-config\") pod \"kube-apiserver-proxy-ip-10-0-129-16.ec2.internal\" (UID: \"3b8d2414fddfe8167462c801d0be4b0d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.920562 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.920545    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ba3a9b5625ee0077a9baad34e458c51-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal\" (UID: \"7ba3a9b5625ee0077a9baad34e458c51\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:43.920562 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:43.920557    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ba3a9b5625ee0077a9baad34e458c51-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal\" (UID: \"7ba3a9b5625ee0077a9baad34e458c51\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:45.218336 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:45.218287    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:45.263792 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:45.263759    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:45.268864 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:45.268845    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:45.319110 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:45.319079    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:45.419678 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:45.419636    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:45.520270 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:45.520193    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:45.620854 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:45.620823    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:45.663347 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:45.663322    2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 20 22:22:45.663493 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:45.663476    2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 22:22:45.721745 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:45.721697    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:45.739546 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:45.739519    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 22:17:44 +0000 UTC" deadline="2028-04-02 21:56:56.548325872 +0000 UTC"
Jul 20 22:22:45.739546 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:45.739544    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14927h34m10.808784721s"
Jul 20 22:22:45.810384 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:45.810309    2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 22:22:45.822628 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:45.822597    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:45.923500 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:45.923466    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:45.940058 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:45.940018    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b8d2414fddfe8167462c801d0be4b0d.slice/crio-bc979b55330400bc2f0cff8bd58c455de07aa188d981208819f9b9a3b4638f63 WatchSource:0}: Error finding container bc979b55330400bc2f0cff8bd58c455de07aa188d981208819f9b9a3b4638f63: Status 404 returned error can't find the container with id bc979b55330400bc2f0cff8bd58c455de07aa188d981208819f9b9a3b4638f63
Jul 20 22:22:45.940225 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:45.940208    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ba3a9b5625ee0077a9baad34e458c51.slice/crio-117f7697a5005a8d46d224673e242cbcf8475b0e3263aa5906e643143accfbc6 WatchSource:0}: Error finding container 117f7697a5005a8d46d224673e242cbcf8475b0e3263aa5906e643143accfbc6: Status 404 returned error can't find the container with id 117f7697a5005a8d46d224673e242cbcf8475b0e3263aa5906e643143accfbc6
Jul 20 22:22:45.945615 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:45.945590    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 22:22:46.024501 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.024461    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:46.125020 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.124947    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:46.225661 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.225632    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-16.ec2.internal\" not found"
Jul 20 22:22:46.242233 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.242210    2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 22:22:46.306007 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.305978    2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 22:22:46.324536 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.324508    2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:46.334305 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.334283    2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 22:22:46.335310 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.335288    2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal"
Jul 20 22:22:46.344130 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.344111    2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 22:22:46.703083 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.703051    2550 apiserver.go:52] "Watching apiserver"
Jul 20 22:22:46.708836 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.708806    2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 20 22:22:46.711733 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.711692    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc","openshift-image-registry/node-ca-7vxbl","openshift-network-diagnostics/network-check-target-tzv8r","openshift-network-operator/iptables-alerter-x8hch","kube-system/konnectivity-agent-85xd4","openshift-cluster-node-tuning-operator/tuned-vnqk2","openshift-dns/node-resolver-bw9fm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal","openshift-multus/multus-additional-cni-plugins-g7tg5","openshift-multus/multus-xsrgm","openshift-multus/network-metrics-daemon-m5bvh","openshift-ovn-kubernetes/ovnkube-node-h8985","kube-system/global-pull-secret-syncer-89dp7"]
Jul 20 22:22:46.713806 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.713788    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.716410 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.716391    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.716509 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.716439    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-k8c86\""
Jul 20 22:22:46.716509 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.716464    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 20 22:22:46.716509 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.716495    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 20 22:22:46.718017 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.717995    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7vxbl"
Jul 20 22:22:46.719412 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.718791    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8k49j\""
Jul 20 22:22:46.719412 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.718987    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 20 22:22:46.719412 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.719045    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 20 22:22:46.719412 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.719254    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 20 22:22:46.719412 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.719287    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 20 22:22:46.719703 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.719687    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:22:46.719800 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.719773    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:22:46.719906 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.719890    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 20 22:22:46.720753 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.720659    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 20 22:22:46.720864 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.720771    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 20 22:22:46.721256 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.721224    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k4rjq\""
Jul 20 22:22:46.723032 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.722048    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x8hch"
Jul 20 22:22:46.724080 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.724053    2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 20 22:22:46.724386 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.724359    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 20 22:22:46.724591 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.724575    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 20 22:22:46.724753 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.724579    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8lk9t\""
Jul 20 22:22:46.724896 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.724366    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 20 22:22:46.726560 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.726037    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-85xd4"
Jul 20 22:22:46.727678 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.727659    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:46.727826 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.727801    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:22:46.728110 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.728092    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-r257t\""
Jul 20 22:22:46.728275 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.728259    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 20 22:22:46.728328 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.728275    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 20 22:22:46.729823 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.729760    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.731844 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.731811    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.732095 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.732075    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 20 22:22:46.732183 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.732165    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tpc5w\""
Jul 20 22:22:46.732242 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.732188    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 20 22:22:46.733813 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.733794    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-njxxd\""
Jul 20 22:22:46.734069 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.734054    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 20 22:22:46.734292 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.734098    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 20 22:22:46.734365 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.734335    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 20 22:22:46.734509 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.734439    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.735991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.735769    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79bb8\" (UniqueName: \"kubernetes.io/projected/5d198525-03ef-41a7-80f8-ab1f2fff16ca-kube-api-access-79bb8\") pod \"iptables-alerter-x8hch\" (UID: \"5d198525-03ef-41a7-80f8-ab1f2fff16ca\") " pod="openshift-network-operator/iptables-alerter-x8hch"
Jul 20 22:22:46.735991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.735829    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-sysctl-d\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.735991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.735851    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-run\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.735991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.735873    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-run-netns\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.735991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.735915    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-var-lib-cni-bin\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.735991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.735948    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-var-lib-kubelet\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.735991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.735968    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-multus-conf-dir\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.736396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.735999    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d198525-03ef-41a7-80f8-ab1f2fff16ca-host-slash\") pod \"iptables-alerter-x8hch\" (UID: \"5d198525-03ef-41a7-80f8-ab1f2fff16ca\") " pod="openshift-network-operator/iptables-alerter-x8hch"
Jul 20 22:22:46.736396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736023    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-system-cni-dir\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.736396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736057    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-multus-cni-dir\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.736396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736079    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-cni-binary-copy\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.736396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736148    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7dj7\" (UniqueName: \"kubernetes.io/projected/07ea4c4b-623c-44c9-a630-22f2cf3f91f0-kube-api-access-v7dj7\") pod \"node-ca-7vxbl\" (UID: \"07ea4c4b-623c-44c9-a630-22f2cf3f91f0\") " pod="openshift-image-registry/node-ca-7vxbl"
Jul 20 22:22:46.736396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736177    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-modprobe-d\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.736396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736200    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-kubernetes\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.736396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736233    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-multus-socket-dir-parent\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.736396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736274    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-run-multus-certs\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.736396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736309    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-etc-kubernetes\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.736396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736340    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07ea4c4b-623c-44c9-a630-22f2cf3f91f0-serviceca\") pod \"node-ca-7vxbl\" (UID: \"07ea4c4b-623c-44c9-a630-22f2cf3f91f0\") " pod="openshift-image-registry/node-ca-7vxbl"
Jul 20 22:22:46.736396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736364    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d198525-03ef-41a7-80f8-ab1f2fff16ca-iptables-alerter-script\") pod \"iptables-alerter-x8hch\" (UID: \"5d198525-03ef-41a7-80f8-ab1f2fff16ca\") " pod="openshift-network-operator/iptables-alerter-x8hch"
Jul 20 22:22:46.736396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736388    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-lib-modules\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736419    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-var-lib-cni-multus\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736443    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fprgn\" (UniqueName: \"kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn\") pod \"network-check-target-tzv8r\" (UID: \"63b300d3-efba-49ec-8760-7b84c44010d2\") " pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736464    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-tmp\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736489    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2b9d8d5a-7d89-4556-b963-ef432f0b749e-konnectivity-ca\") pod \"konnectivity-agent-85xd4\" (UID: \"2b9d8d5a-7d89-4556-b963-ef432f0b749e\") " pod="kube-system/konnectivity-agent-85xd4"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736511    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-var-lib-kubelet\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736532    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-tuned\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736556    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mm4j\" (UniqueName: \"kubernetes.io/projected/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-kube-api-access-8mm4j\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736576    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07ea4c4b-623c-44c9-a630-22f2cf3f91f0-host\") pod \"node-ca-7vxbl\" (UID: \"07ea4c4b-623c-44c9-a630-22f2cf3f91f0\") " pod="openshift-image-registry/node-ca-7vxbl"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736597    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-host\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736628    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-cnibin\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736651    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-os-release\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736702    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-run-k8s-cni-cncf-io\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736776    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736766    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-multus-daemon-config\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736833    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q9bz7\""
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736814    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736812    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736902    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736982    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bw9fm"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736842    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-sysctl-conf\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.737107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.736851    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 20 22:22:46.738119 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.737041    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-systemd\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.738119 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.737065    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2b9d8d5a-7d89-4556-b963-ef432f0b749e-agent-certs\") pod \"konnectivity-agent-85xd4\" (UID: \"2b9d8d5a-7d89-4556-b963-ef432f0b749e\") " pod="kube-system/konnectivity-agent-85xd4"
Jul 20 22:22:46.738119 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.737087    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-sys\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.738119 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.737130    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-sysconfig\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.738119 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.737154    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsvrh\" (UniqueName: \"kubernetes.io/projected/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-kube-api-access-zsvrh\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.738119 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.737161    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 20 22:22:46.738119 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.737194    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-hostroot\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.739004 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.738967    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:46.739088 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.739024    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 20 22:22:46.739088 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.739067    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:22:46.739197 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.738968    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v8ljg\""
Jul 20 22:22:46.739197 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.739153    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 20 22:22:46.739566 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.739412    2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 22:22:46.763942 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.763910    2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ndb84"
Jul 20 22:22:46.771932 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.771909    2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ndb84"
Jul 20 22:22:46.825073 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.825041    2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 20 22:22:46.837643 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.837616    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fprgn\" (UniqueName: \"kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn\") pod \"network-check-target-tzv8r\" (UID: \"63b300d3-efba-49ec-8760-7b84c44010d2\") " pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:22:46.837802 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.837656    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-registration-dir\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.837802 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.837705    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-run-systemd\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.837802 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.837741    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgqh7\" (UniqueName: \"kubernetes.io/projected/b94e391c-6611-4879-a037-77852e20a2af-kube-api-access-qgqh7\") pod \"node-resolver-bw9fm\" (UID: \"b94e391c-6611-4879-a037-77852e20a2af\") " pod="openshift-dns/node-resolver-bw9fm"
Jul 20 22:22:46.837951 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.837792    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-var-lib-kubelet\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.837951 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.837826    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-tuned\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.837951 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.837843    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mm4j\" (UniqueName: \"kubernetes.io/projected/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-kube-api-access-8mm4j\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.837951 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.837858    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07ea4c4b-623c-44c9-a630-22f2cf3f91f0-host\") pod \"node-ca-7vxbl\" (UID: \"07ea4c4b-623c-44c9-a630-22f2cf3f91f0\") " pod="openshift-image-registry/node-ca-7vxbl"
Jul 20 22:22:46.837951 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.837892    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-var-lib-kubelet\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.837951 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.837898    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07ea4c4b-623c-44c9-a630-22f2cf3f91f0-host\") pod \"node-ca-7vxbl\" (UID: \"07ea4c4b-623c-44c9-a630-22f2cf3f91f0\") " pod="openshift-image-registry/node-ca-7vxbl"
Jul 20 22:22:46.837951 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.837930    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:46.838246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.837969    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-os-release\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.838246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.837992    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-slash\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.838246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838009    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-run-openvswitch\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.838246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838025    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6cl7\" (UniqueName: \"kubernetes.io/projected/87b10474-5ce4-419f-bf63-d9d4b32e7861-kube-api-access-p6cl7\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.838246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838047    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-systemd\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.838246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838088    2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 20 22:22:46.838246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838096    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-os-release\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.838246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838089    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1c875ab-aa59-4235-ac20-201d722fcc7f-system-cni-dir\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.838246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838142    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-node-log\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.838246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838147    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-systemd\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.838246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838165    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87b10474-5ce4-419f-bf63-d9d4b32e7861-ovn-node-metrics-cert\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.838246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838182    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-sysctl-d\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.838246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838239    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-run\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838263    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-var-lib-kubelet\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838285    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-multus-conf-dir\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838318    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-multus-conf-dir\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838324    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-run\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838329    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-var-lib-kubelet\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838349    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-sysctl-d\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838371    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-sys-fs\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838398    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-log-socket\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838422    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-cni-netd\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838449    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7dj7\" (UniqueName: \"kubernetes.io/projected/07ea4c4b-623c-44c9-a630-22f2cf3f91f0-kube-api-access-v7dj7\") pod \"node-ca-7vxbl\" (UID: \"07ea4c4b-623c-44c9-a630-22f2cf3f91f0\") " pod="openshift-image-registry/node-ca-7vxbl"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838475    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e1c875ab-aa59-4235-ac20-201d722fcc7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838501    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-device-dir\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838528    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-systemd-units\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838551    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-etc-openvswitch\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838580    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-modprobe-d\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838606    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-kubernetes\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.838885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838638    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-multus-socket-dir-parent\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838663    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1c875ab-aa59-4235-ac20-201d722fcc7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838691    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-kubernetes\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838693    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-run-netns\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838708    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-modprobe-d\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838750    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-run-ovn-kubernetes\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838777    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-dbus\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838785    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-multus-socket-dir-parent\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838794    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-lib-modules\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838828    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-var-lib-cni-multus\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838856    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1c875ab-aa59-4235-ac20-201d722fcc7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838871    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-lib-modules\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838880    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-etc-selinux\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838903    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-cni-bin\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838906    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-var-lib-cni-multus\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838928    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-tmp\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838954    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2b9d8d5a-7d89-4556-b963-ef432f0b749e-konnectivity-ca\") pod \"konnectivity-agent-85xd4\" (UID: \"2b9d8d5a-7d89-4556-b963-ef432f0b749e\") " pod="kube-system/konnectivity-agent-85xd4"
Jul 20 22:22:46.839666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838978    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-host\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.838999    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-cnibin\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839029    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-run-k8s-cni-cncf-io\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839047    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-host\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839059    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-cnibin\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839086    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-multus-daemon-config\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839097    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-run-k8s-cni-cncf-io\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839113    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1c875ab-aa59-4235-ac20-201d722fcc7f-cnibin\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839139    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839163    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87b10474-5ce4-419f-bf63-d9d4b32e7861-ovnkube-config\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839190    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-sysctl-conf\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839217    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2b9d8d5a-7d89-4556-b963-ef432f0b749e-agent-certs\") pod \"konnectivity-agent-85xd4\" (UID: \"2b9d8d5a-7d89-4556-b963-ef432f0b749e\") " pod="kube-system/konnectivity-agent-85xd4"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839241    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-sys\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839266    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839292    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lffk\" (UniqueName: \"kubernetes.io/projected/116caf25-c9e5-47da-a793-bf52c4aa1f5e-kube-api-access-2lffk\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839303    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-sysctl-conf\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839315    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1c875ab-aa59-4235-ac20-201d722fcc7f-os-release\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839339    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-socket-dir\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839370    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-var-lib-openvswitch\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839395    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-sys\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839430    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-sysconfig\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839461    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsvrh\" (UniqueName: \"kubernetes.io/projected/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-kube-api-access-zsvrh\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839489    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-sysconfig\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839526    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-hostroot\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839558    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-run-ovn\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839604    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-hostroot\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839626    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-multus-daemon-config\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839631    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87b10474-5ce4-419f-bf63-d9d4b32e7861-env-overrides\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839657    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b94e391c-6611-4879-a037-77852e20a2af-hosts-file\") pod \"node-resolver-bw9fm\" (UID: \"b94e391c-6611-4879-a037-77852e20a2af\") " pod="openshift-dns/node-resolver-bw9fm"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839787    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b94e391c-6611-4879-a037-77852e20a2af-tmp-dir\") pod \"node-resolver-bw9fm\" (UID: \"b94e391c-6611-4879-a037-77852e20a2af\") " pod="openshift-dns/node-resolver-bw9fm"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839821    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79bb8\" (UniqueName: \"kubernetes.io/projected/5d198525-03ef-41a7-80f8-ab1f2fff16ca-kube-api-access-79bb8\") pod \"iptables-alerter-x8hch\" (UID: \"5d198525-03ef-41a7-80f8-ab1f2fff16ca\") " pod="openshift-network-operator/iptables-alerter-x8hch"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839864    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-run-netns\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839889    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-var-lib-cni-bin\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839916    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e1c875ab-aa59-4235-ac20-201d722fcc7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.840912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839944    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839950    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-var-lib-cni-bin\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839917    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-run-netns\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839979    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2b9d8d5a-7d89-4556-b963-ef432f0b749e-konnectivity-ca\") pod \"konnectivity-agent-85xd4\" (UID: \"2b9d8d5a-7d89-4556-b963-ef432f0b749e\") " pod="kube-system/konnectivity-agent-85xd4"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.839999    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87b10474-5ce4-419f-bf63-d9d4b32e7861-ovnkube-script-lib\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840026    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-kubelet-config\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840068    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d198525-03ef-41a7-80f8-ab1f2fff16ca-host-slash\") pod \"iptables-alerter-x8hch\" (UID: \"5d198525-03ef-41a7-80f8-ab1f2fff16ca\") " pod="openshift-network-operator/iptables-alerter-x8hch"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840104    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-system-cni-dir\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840158    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d198525-03ef-41a7-80f8-ab1f2fff16ca-host-slash\") pod \"iptables-alerter-x8hch\" (UID: \"5d198525-03ef-41a7-80f8-ab1f2fff16ca\") " pod="openshift-network-operator/iptables-alerter-x8hch"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840162    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-system-cni-dir\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840197    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-multus-cni-dir\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840218    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-cni-binary-copy\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840245    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pmwj\" (UniqueName: \"kubernetes.io/projected/e1c875ab-aa59-4235-ac20-201d722fcc7f-kube-api-access-5pmwj\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840266    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-kubelet\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840287    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-run-multus-certs\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840308    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-etc-kubernetes\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840331    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07ea4c4b-623c-44c9-a630-22f2cf3f91f0-serviceca\") pod \"node-ca-7vxbl\" (UID: \"07ea4c4b-623c-44c9-a630-22f2cf3f91f0\") " pod="openshift-image-registry/node-ca-7vxbl"
Jul 20 22:22:46.841648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840348    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-multus-cni-dir\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.842389 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840353    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nsqg\" (UniqueName: \"kubernetes.io/projected/01eff038-0ad7-4c30-b246-52001c256b89-kube-api-access-9nsqg\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.842389 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840516    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d198525-03ef-41a7-80f8-ab1f2fff16ca-iptables-alerter-script\") pod \"iptables-alerter-x8hch\" (UID: \"5d198525-03ef-41a7-80f8-ab1f2fff16ca\") " pod="openshift-network-operator/iptables-alerter-x8hch"
Jul 20 22:22:46.842389 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840656    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-host-run-multus-certs\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.842389 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840752    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-etc-kubernetes\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.842389 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.840842    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-cni-binary-copy\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.842389 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.841153    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07ea4c4b-623c-44c9-a630-22f2cf3f91f0-serviceca\") pod \"node-ca-7vxbl\" (UID: \"07ea4c4b-623c-44c9-a630-22f2cf3f91f0\") " pod="openshift-image-registry/node-ca-7vxbl"
Jul 20 22:22:46.842389 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.841159    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d198525-03ef-41a7-80f8-ab1f2fff16ca-iptables-alerter-script\") pod \"iptables-alerter-x8hch\" (UID: \"5d198525-03ef-41a7-80f8-ab1f2fff16ca\") " pod="openshift-network-operator/iptables-alerter-x8hch"
Jul 20 22:22:46.842389 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.841632    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-tmp\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.842389 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.841676    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-etc-tuned\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.842805 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.842588    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2b9d8d5a-7d89-4556-b963-ef432f0b749e-agent-certs\") pod \"konnectivity-agent-85xd4\" (UID: \"2b9d8d5a-7d89-4556-b963-ef432f0b749e\") " pod="kube-system/konnectivity-agent-85xd4"
Jul 20 22:22:46.843234 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.843178    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal" event={"ID":"7ba3a9b5625ee0077a9baad34e458c51","Type":"ContainerStarted","Data":"117f7697a5005a8d46d224673e242cbcf8475b0e3263aa5906e643143accfbc6"}
Jul 20 22:22:46.844392 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.844366    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal" event={"ID":"3b8d2414fddfe8167462c801d0be4b0d","Type":"ContainerStarted","Data":"bc979b55330400bc2f0cff8bd58c455de07aa188d981208819f9b9a3b4638f63"}
Jul 20 22:22:46.845293 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.845238    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 22:22:46.845293 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.845262    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 22:22:46.845293 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.845274    2550 projected.go:196] Error preparing data for projected volume kube-api-access-fprgn for pod openshift-network-diagnostics/network-check-target-tzv8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:22:46.845494 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.845398    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn podName:63b300d3-efba-49ec-8760-7b84c44010d2 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:47.345331135 +0000 UTC m=+3.123035178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fprgn" (UniqueName: "kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn") pod "network-check-target-tzv8r" (UID: "63b300d3-efba-49ec-8760-7b84c44010d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:22:46.847080 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.847053    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7dj7\" (UniqueName: \"kubernetes.io/projected/07ea4c4b-623c-44c9-a630-22f2cf3f91f0-kube-api-access-v7dj7\") pod \"node-ca-7vxbl\" (UID: \"07ea4c4b-623c-44c9-a630-22f2cf3f91f0\") " pod="openshift-image-registry/node-ca-7vxbl"
Jul 20 22:22:46.847219 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.847204    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79bb8\" (UniqueName: \"kubernetes.io/projected/5d198525-03ef-41a7-80f8-ab1f2fff16ca-kube-api-access-79bb8\") pod \"iptables-alerter-x8hch\" (UID: \"5d198525-03ef-41a7-80f8-ab1f2fff16ca\") " pod="openshift-network-operator/iptables-alerter-x8hch"
Jul 20 22:22:46.847523 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.847509    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mm4j\" (UniqueName: \"kubernetes.io/projected/2c4d3266-7aaa-4d42-952c-f98e4e29dec4-kube-api-access-8mm4j\") pod \"multus-xsrgm\" (UID: \"2c4d3266-7aaa-4d42-952c-f98e4e29dec4\") " pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:46.847626 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.847612    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsvrh\" (UniqueName: \"kubernetes.io/projected/2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74-kube-api-access-zsvrh\") pod \"tuned-vnqk2\" (UID: \"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74\") " pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:46.941203 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941172    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e1c875ab-aa59-4235-ac20-201d722fcc7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.941203 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941204    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.941420 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941241    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87b10474-5ce4-419f-bf63-d9d4b32e7861-ovnkube-script-lib\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.941420 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941267    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-kubelet-config\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:46.941420 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941294    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pmwj\" (UniqueName: \"kubernetes.io/projected/e1c875ab-aa59-4235-ac20-201d722fcc7f-kube-api-access-5pmwj\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.941420 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941299    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.941420 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941318    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-kubelet\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.941420 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941378    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-kubelet-config\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:46.941420 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941396    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nsqg\" (UniqueName: \"kubernetes.io/projected/01eff038-0ad7-4c30-b246-52001c256b89-kube-api-access-9nsqg\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.941420 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941417    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-kubelet\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941454    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-registration-dir\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941479    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-run-systemd\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941507    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgqh7\" (UniqueName: \"kubernetes.io/projected/b94e391c-6611-4879-a037-77852e20a2af-kube-api-access-qgqh7\") pod \"node-resolver-bw9fm\" (UID: \"b94e391c-6611-4879-a037-77852e20a2af\") " pod="openshift-dns/node-resolver-bw9fm"
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941539    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941541    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-registration-dir\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941564    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-slash\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941584    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-run-systemd\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941588    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-run-openvswitch\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941628    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6cl7\" (UniqueName: \"kubernetes.io/projected/87b10474-5ce4-419f-bf63-d9d4b32e7861-kube-api-access-p6cl7\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941628    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-run-openvswitch\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.941707    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941737    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-slash\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.941783    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret podName:d7df0b06-2bc3-45b8-8443-6c4583b2ca28 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:47.441766575 +0000 UTC m=+3.219470605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret") pod "global-pull-secret-syncer-89dp7" (UID: "d7df0b06-2bc3-45b8-8443-6c4583b2ca28") : object "kube-system"/"original-pull-secret" not registered
Jul 20 22:22:46.941822 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941806    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1c875ab-aa59-4235-ac20-201d722fcc7f-system-cni-dir\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941835    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-node-log\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941845    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e1c875ab-aa59-4235-ac20-201d722fcc7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941860    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87b10474-5ce4-419f-bf63-d9d4b32e7861-ovn-node-metrics-cert\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941890    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-sys-fs\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941902    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-node-log\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941890    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1c875ab-aa59-4235-ac20-201d722fcc7f-system-cni-dir\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941945    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-sys-fs\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.941969    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87b10474-5ce4-419f-bf63-d9d4b32e7861-ovnkube-script-lib\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942026    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-log-socket\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942090    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-log-socket\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942093    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-cni-netd\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942137    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-cni-netd\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942163    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e1c875ab-aa59-4235-ac20-201d722fcc7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942220    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-device-dir\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942252    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-systemd-units\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942277    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-etc-openvswitch\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.942472 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942301    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1c875ab-aa59-4235-ac20-201d722fcc7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942347    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-run-netns\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942349    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-systemd-units\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942372    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-run-ovn-kubernetes\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942352    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-etc-openvswitch\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942397    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-dbus\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942446    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-device-dir\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942437    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-run-netns\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942481    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1c875ab-aa59-4235-ac20-201d722fcc7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942481    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-run-ovn-kubernetes\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942542    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-dbus\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942548    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-etc-selinux\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942584    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-cni-bin\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942614    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1c875ab-aa59-4235-ac20-201d722fcc7f-cnibin\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942651    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942693    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1c875ab-aa59-4235-ac20-201d722fcc7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942707    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87b10474-5ce4-419f-bf63-d9d4b32e7861-ovnkube-config\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.943266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942742    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-etc-selinux\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942775    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1c875ab-aa59-4235-ac20-201d722fcc7f-cnibin\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942781    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942805    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942816    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-host-cni-bin\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942810    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lffk\" (UniqueName: \"kubernetes.io/projected/116caf25-c9e5-47da-a793-bf52c4aa1f5e-kube-api-access-2lffk\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942853    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1c875ab-aa59-4235-ac20-201d722fcc7f-os-release\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942863    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e1c875ab-aa59-4235-ac20-201d722fcc7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942888    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-socket-dir\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.942922    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942929    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1c875ab-aa59-4235-ac20-201d722fcc7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:46.942965    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs podName:116caf25-c9e5-47da-a793-bf52c4aa1f5e nodeName:}" failed. No retries permitted until 2026-07-20 22:22:47.442949581 +0000 UTC m=+3.220653625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs") pod "network-metrics-daemon-m5bvh" (UID: "116caf25-c9e5-47da-a793-bf52c4aa1f5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942953    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1c875ab-aa59-4235-ac20-201d722fcc7f-os-release\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.942986    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/01eff038-0ad7-4c30-b246-52001c256b89-socket-dir\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.943001    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-var-lib-openvswitch\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.943031    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-run-ovn\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.943054    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87b10474-5ce4-419f-bf63-d9d4b32e7861-env-overrides\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.943952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.943081    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b94e391c-6611-4879-a037-77852e20a2af-hosts-file\") pod \"node-resolver-bw9fm\" (UID: \"b94e391c-6611-4879-a037-77852e20a2af\") " pod="openshift-dns/node-resolver-bw9fm"
Jul 20 22:22:46.944488 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.943093    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-var-lib-openvswitch\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.944488 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.943104    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b94e391c-6611-4879-a037-77852e20a2af-tmp-dir\") pod \"node-resolver-bw9fm\" (UID: \"b94e391c-6611-4879-a037-77852e20a2af\") " pod="openshift-dns/node-resolver-bw9fm"
Jul 20 22:22:46.944488 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.943160    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87b10474-5ce4-419f-bf63-d9d4b32e7861-run-ovn\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.944488 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.943234    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b94e391c-6611-4879-a037-77852e20a2af-hosts-file\") pod \"node-resolver-bw9fm\" (UID: \"b94e391c-6611-4879-a037-77852e20a2af\") " pod="openshift-dns/node-resolver-bw9fm"
Jul 20 22:22:46.944488 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.943388    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87b10474-5ce4-419f-bf63-d9d4b32e7861-ovnkube-config\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.944488 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.943420    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b94e391c-6611-4879-a037-77852e20a2af-tmp-dir\") pod \"node-resolver-bw9fm\" (UID: \"b94e391c-6611-4879-a037-77852e20a2af\") " pod="openshift-dns/node-resolver-bw9fm"
Jul 20 22:22:46.944488 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.943503    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87b10474-5ce4-419f-bf63-d9d4b32e7861-env-overrides\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.944771 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.944498    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87b10474-5ce4-419f-bf63-d9d4b32e7861-ovn-node-metrics-cert\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.953039 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.951042    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nsqg\" (UniqueName: \"kubernetes.io/projected/01eff038-0ad7-4c30-b246-52001c256b89-kube-api-access-9nsqg\") pod \"aws-ebs-csi-driver-node-tnzsc\" (UID: \"01eff038-0ad7-4c30-b246-52001c256b89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:46.953039 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.951453    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgqh7\" (UniqueName: \"kubernetes.io/projected/b94e391c-6611-4879-a037-77852e20a2af-kube-api-access-qgqh7\") pod \"node-resolver-bw9fm\" (UID: \"b94e391c-6611-4879-a037-77852e20a2af\") " pod="openshift-dns/node-resolver-bw9fm"
Jul 20 22:22:46.953039 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.951531    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6cl7\" (UniqueName: \"kubernetes.io/projected/87b10474-5ce4-419f-bf63-d9d4b32e7861-kube-api-access-p6cl7\") pod \"ovnkube-node-h8985\" (UID: \"87b10474-5ce4-419f-bf63-d9d4b32e7861\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:46.953039 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.952285    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pmwj\" (UniqueName: \"kubernetes.io/projected/e1c875ab-aa59-4235-ac20-201d722fcc7f-kube-api-access-5pmwj\") pod \"multus-additional-cni-plugins-g7tg5\" (UID: \"e1c875ab-aa59-4235-ac20-201d722fcc7f\") " pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:46.954813 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:46.954766    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lffk\" (UniqueName: \"kubernetes.io/projected/116caf25-c9e5-47da-a793-bf52c4aa1f5e-kube-api-access-2lffk\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:47.010870 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.010838    2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 22:22:47.026970 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.026943    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vnqk2"
Jul 20 22:22:47.035173 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.035149    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xsrgm"
Jul 20 22:22:47.046828 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.046800    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7vxbl"
Jul 20 22:22:47.052518 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.052489    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x8hch"
Jul 20 22:22:47.060076 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.060053    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-85xd4"
Jul 20 22:22:47.067554 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.067535    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g7tg5"
Jul 20 22:22:47.075179 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.075157    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc"
Jul 20 22:22:47.081905 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.081883    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:22:47.088572 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.088553    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bw9fm"
Jul 20 22:22:47.346110 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.346025    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fprgn\" (UniqueName: \"kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn\") pod \"network-check-target-tzv8r\" (UID: \"63b300d3-efba-49ec-8760-7b84c44010d2\") " pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:22:47.346274 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:47.346198    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 22:22:47.346274 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:47.346223    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 22:22:47.346274 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:47.346236    2550 projected.go:196] Error preparing data for projected volume kube-api-access-fprgn for pod openshift-network-diagnostics/network-check-target-tzv8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:22:47.346425 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:47.346299    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn podName:63b300d3-efba-49ec-8760-7b84c44010d2 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:48.346280385 +0000 UTC m=+4.123984416 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fprgn" (UniqueName: "kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn") pod "network-check-target-tzv8r" (UID: "63b300d3-efba-49ec-8760-7b84c44010d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:22:47.446477 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.446414    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:47.446672 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.446486    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:47.446672 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:47.446584    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 22:22:47.446672 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:47.446599    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:22:47.446672 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:47.446653    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret podName:d7df0b06-2bc3-45b8-8443-6c4583b2ca28 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:48.446632824 +0000 UTC m=+4.224336859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret") pod "global-pull-secret-syncer-89dp7" (UID: "d7df0b06-2bc3-45b8-8443-6c4583b2ca28") : object "kube-system"/"original-pull-secret" not registered
Jul 20 22:22:47.446672 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:47.446669    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs podName:116caf25-c9e5-47da-a793-bf52c4aa1f5e nodeName:}" failed. No retries permitted until 2026-07-20 22:22:48.446662206 +0000 UTC m=+4.224366235 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs") pod "network-metrics-daemon-m5bvh" (UID: "116caf25-c9e5-47da-a793-bf52c4aa1f5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:22:47.775399 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.775270    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 22:17:46 +0000 UTC" deadline="2028-02-07 21:48:39.634790096 +0000 UTC"
Jul 20 22:22:47.775399 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.775338    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13607h25m51.859455622s"
Jul 20 22:22:47.838890 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:47.838866    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:47.839041 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:47.839018    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:22:47.851532 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:47.851501    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c875ab_aa59_4235_ac20_201d722fcc7f.slice/crio-1c40906f4e6668e2778450e6237a0f8ba2eabd05ea8485d04929da761aaaacc3 WatchSource:0}: Error finding container 1c40906f4e6668e2778450e6237a0f8ba2eabd05ea8485d04929da761aaaacc3: Status 404 returned error can't find the container with id 1c40906f4e6668e2778450e6237a0f8ba2eabd05ea8485d04929da761aaaacc3
Jul 20 22:22:47.853895 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:47.853868    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b9d8d5a_7d89_4556_b963_ef432f0b749e.slice/crio-3f1d421f5837130c4f0d4acb50fd2ab6b0c3dab3c12500c9892a5be9da4395d1 WatchSource:0}: Error finding container 3f1d421f5837130c4f0d4acb50fd2ab6b0c3dab3c12500c9892a5be9da4395d1: Status 404 returned error can't find the container with id 3f1d421f5837130c4f0d4acb50fd2ab6b0c3dab3c12500c9892a5be9da4395d1
Jul 20 22:22:47.854942 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:47.854917    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94e391c_6611_4879_a037_77852e20a2af.slice/crio-33c4341bc755af7f529ae070f6d6f1d0754e4cac48bb5c22b359d563315d50f1 WatchSource:0}: Error finding container 33c4341bc755af7f529ae070f6d6f1d0754e4cac48bb5c22b359d563315d50f1: Status 404 returned error can't find the container with id 33c4341bc755af7f529ae070f6d6f1d0754e4cac48bb5c22b359d563315d50f1
Jul 20 22:22:47.878710 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:47.878687    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ea4c4b_623c_44c9_a630_22f2cf3f91f0.slice/crio-bc8d93e321b508e4745d3ebb21a4ad71bb4e33be3ed144b5d1476f84b813381a WatchSource:0}: Error finding container bc8d93e321b508e4745d3ebb21a4ad71bb4e33be3ed144b5d1476f84b813381a: Status 404 returned error can't find the container with id bc8d93e321b508e4745d3ebb21a4ad71bb4e33be3ed144b5d1476f84b813381a
Jul 20 22:22:47.879499 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:47.879413    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c4d3266_7aaa_4d42_952c_f98e4e29dec4.slice/crio-7560955ad01016c437bb6bd5986d0db15ad2f1260b116d0b80926276655a2a34 WatchSource:0}: Error finding container 7560955ad01016c437bb6bd5986d0db15ad2f1260b116d0b80926276655a2a34: Status 404 returned error can't find the container with id 7560955ad01016c437bb6bd5986d0db15ad2f1260b116d0b80926276655a2a34
Jul 20 22:22:47.880053 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:47.880003    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01eff038_0ad7_4c30_b246_52001c256b89.slice/crio-6cf2f34fe911b66462ac6d7a3b2f6ab30a225780faf15f2afb994da0bf9d2831 WatchSource:0}: Error finding container 6cf2f34fe911b66462ac6d7a3b2f6ab30a225780faf15f2afb994da0bf9d2831: Status 404 returned error can't find the container with id 6cf2f34fe911b66462ac6d7a3b2f6ab30a225780faf15f2afb994da0bf9d2831
Jul 20 22:22:47.880884 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:47.880861    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a64e9d5_1c6a_40e0_9d99_9d3ffabaad74.slice/crio-1e03444a2b47d26d9485c593c4d21a15cee4203f8935334257d98c343937c9b2 WatchSource:0}: Error finding container 1e03444a2b47d26d9485c593c4d21a15cee4203f8935334257d98c343937c9b2: Status 404 returned error can't find the container with id 1e03444a2b47d26d9485c593c4d21a15cee4203f8935334257d98c343937c9b2
Jul 20 22:22:47.882410 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:47.882391    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87b10474_5ce4_419f_bf63_d9d4b32e7861.slice/crio-2eb08eecad4b34369019bfe8c5b27648c3a8b0cec98832e01bb38061b301b84c WatchSource:0}: Error finding container 2eb08eecad4b34369019bfe8c5b27648c3a8b0cec98832e01bb38061b301b84c: Status 404 returned error can't find the container with id 2eb08eecad4b34369019bfe8c5b27648c3a8b0cec98832e01bb38061b301b84c
Jul 20 22:22:47.885876 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:22:47.885853    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d198525_03ef_41a7_80f8_ab1f2fff16ca.slice/crio-871c280bb7297cb95828ee6ec685d7fd7efffd1e5b1111aa9178ed9ccdf1e510 WatchSource:0}: Error finding container 871c280bb7297cb95828ee6ec685d7fd7efffd1e5b1111aa9178ed9ccdf1e510: Status 404 returned error can't find the container with id 871c280bb7297cb95828ee6ec685d7fd7efffd1e5b1111aa9178ed9ccdf1e510
Jul 20 22:22:48.353571 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.353310    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fprgn\" (UniqueName: \"kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn\") pod \"network-check-target-tzv8r\" (UID: \"63b300d3-efba-49ec-8760-7b84c44010d2\") " pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:22:48.353571 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:48.353509    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 22:22:48.353571 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:48.353571    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 22:22:48.353861 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:48.353587    2550 projected.go:196] Error preparing data for projected volume kube-api-access-fprgn for pod openshift-network-diagnostics/network-check-target-tzv8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:22:48.353861 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:48.353650    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn podName:63b300d3-efba-49ec-8760-7b84c44010d2 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:50.353632963 +0000 UTC m=+6.131337010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fprgn" (UniqueName: "kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn") pod "network-check-target-tzv8r" (UID: "63b300d3-efba-49ec-8760-7b84c44010d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:22:48.454004 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.453955    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:48.454180 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.454040    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:48.454180 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:48.454166    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 22:22:48.454291 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:48.454228    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret podName:d7df0b06-2bc3-45b8-8443-6c4583b2ca28 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:50.454209816 +0000 UTC m=+6.231913847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret") pod "global-pull-secret-syncer-89dp7" (UID: "d7df0b06-2bc3-45b8-8443-6c4583b2ca28") : object "kube-system"/"original-pull-secret" not registered
Jul 20 22:22:48.454628 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:48.454609    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:22:48.454731 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:48.454660    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs podName:116caf25-c9e5-47da-a793-bf52c4aa1f5e nodeName:}" failed. No retries permitted until 2026-07-20 22:22:50.454644615 +0000 UTC m=+6.232348650 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs") pod "network-metrics-daemon-m5bvh" (UID: "116caf25-c9e5-47da-a793-bf52c4aa1f5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:22:48.776593 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.776522    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 22:17:46 +0000 UTC" deadline="2028-03-05 09:33:07.13378377 +0000 UTC"
Jul 20 22:22:48.776593 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.776563    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14243h10m18.357225936s"
Jul 20 22:22:48.847570 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.843440    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:22:48.847570 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:48.843576    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:22:48.847570 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.843659    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:48.847570 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:48.843751    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:22:48.853041 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.852889    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bw9fm" event={"ID":"b94e391c-6611-4879-a037-77852e20a2af","Type":"ContainerStarted","Data":"33c4341bc755af7f529ae070f6d6f1d0754e4cac48bb5c22b359d563315d50f1"}
Jul 20 22:22:48.855164 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.855133    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-85xd4" event={"ID":"2b9d8d5a-7d89-4556-b963-ef432f0b749e","Type":"ContainerStarted","Data":"3f1d421f5837130c4f0d4acb50fd2ab6b0c3dab3c12500c9892a5be9da4395d1"}
Jul 20 22:22:48.863915 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.863879    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal" event={"ID":"3b8d2414fddfe8167462c801d0be4b0d","Type":"ContainerStarted","Data":"458d46fc4ea267be824f78f6a4a7ae3f44343a6feae182a4a41cd781b6eb0b0c"}
Jul 20 22:22:48.869260 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.869178    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vnqk2" event={"ID":"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74","Type":"ContainerStarted","Data":"1e03444a2b47d26d9485c593c4d21a15cee4203f8935334257d98c343937c9b2"}
Jul 20 22:22:48.872899 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.872859    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xsrgm" event={"ID":"2c4d3266-7aaa-4d42-952c-f98e4e29dec4","Type":"ContainerStarted","Data":"7560955ad01016c437bb6bd5986d0db15ad2f1260b116d0b80926276655a2a34"}
Jul 20 22:22:48.881349 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.881314    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7vxbl" event={"ID":"07ea4c4b-623c-44c9-a630-22f2cf3f91f0","Type":"ContainerStarted","Data":"bc8d93e321b508e4745d3ebb21a4ad71bb4e33be3ed144b5d1476f84b813381a"}
Jul 20 22:22:48.888653 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.888590    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g7tg5" event={"ID":"e1c875ab-aa59-4235-ac20-201d722fcc7f","Type":"ContainerStarted","Data":"1c40906f4e6668e2778450e6237a0f8ba2eabd05ea8485d04929da761aaaacc3"}
Jul 20 22:22:48.900890 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.900847    2550 generic.go:356] "Generic (PLEG): container finished" podID="7ba3a9b5625ee0077a9baad34e458c51" containerID="b6aa8d1db7d2275125965a66e7ac88eba590836cc9e15368488e247c833109b3" exitCode=0
Jul 20 22:22:48.901058 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.900964    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal" event={"ID":"7ba3a9b5625ee0077a9baad34e458c51","Type":"ContainerDied","Data":"b6aa8d1db7d2275125965a66e7ac88eba590836cc9e15368488e247c833109b3"}
Jul 20 22:22:48.914628 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.914352    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x8hch" event={"ID":"5d198525-03ef-41a7-80f8-ab1f2fff16ca","Type":"ContainerStarted","Data":"871c280bb7297cb95828ee6ec685d7fd7efffd1e5b1111aa9178ed9ccdf1e510"}
Jul 20 22:22:48.924550 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.924491    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-16.ec2.internal" podStartSLOduration=2.924472021 podStartE2EDuration="2.924472021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:22:48.883348234 +0000 UTC m=+4.661052288" watchObservedRunningTime="2026-07-20 22:22:48.924472021 +0000 UTC m=+4.702176075"
Jul 20 22:22:48.933555 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.933519    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8985" event={"ID":"87b10474-5ce4-419f-bf63-d9d4b32e7861","Type":"ContainerStarted","Data":"2eb08eecad4b34369019bfe8c5b27648c3a8b0cec98832e01bb38061b301b84c"}
Jul 20 22:22:48.964871 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:48.964756    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc" event={"ID":"01eff038-0ad7-4c30-b246-52001c256b89","Type":"ContainerStarted","Data":"6cf2f34fe911b66462ac6d7a3b2f6ab30a225780faf15f2afb994da0bf9d2831"}
Jul 20 22:22:49.839689 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:49.839024    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:49.839689 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:49.839185    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:22:49.978139 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:49.978091    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal" event={"ID":"7ba3a9b5625ee0077a9baad34e458c51","Type":"ContainerStarted","Data":"21870fa90e33833e17858833fe67e4c51709d2e8502eb66d105c329a63555ad8"}
Jul 20 22:22:50.368926 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:50.368788    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fprgn\" (UniqueName: \"kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn\") pod \"network-check-target-tzv8r\" (UID: \"63b300d3-efba-49ec-8760-7b84c44010d2\") " pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:22:50.369183 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:50.368997    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 22:22:50.369183 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:50.369025    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 22:22:50.369183 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:50.369039    2550 projected.go:196] Error preparing data for projected volume kube-api-access-fprgn for pod openshift-network-diagnostics/network-check-target-tzv8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:22:50.369183 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:50.369101    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn podName:63b300d3-efba-49ec-8760-7b84c44010d2 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:54.36908026 +0000 UTC m=+10.146784293 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fprgn" (UniqueName: "kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn") pod "network-check-target-tzv8r" (UID: "63b300d3-efba-49ec-8760-7b84c44010d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:22:50.469770 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:50.469699    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:50.469961 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:50.469828    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:50.470084 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:50.470036    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:22:50.470142 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:50.470117    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs podName:116caf25-c9e5-47da-a793-bf52c4aa1f5e nodeName:}" failed. No retries permitted until 2026-07-20 22:22:54.470082268 +0000 UTC m=+10.247786301 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs") pod "network-metrics-daemon-m5bvh" (UID: "116caf25-c9e5-47da-a793-bf52c4aa1f5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:22:50.470877 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:50.470800    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 22:22:50.470877 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:50.470856    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret podName:d7df0b06-2bc3-45b8-8443-6c4583b2ca28 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:54.470840378 +0000 UTC m=+10.248544425 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret") pod "global-pull-secret-syncer-89dp7" (UID: "d7df0b06-2bc3-45b8-8443-6c4583b2ca28") : object "kube-system"/"original-pull-secret" not registered
Jul 20 22:22:50.840069 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:50.839384    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:50.840069 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:50.839525    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:22:50.840069 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:50.839880    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:22:50.840069 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:50.839977    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:22:51.839358 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:51.838863    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:51.839358 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:51.838997    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:22:52.839968 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:52.839375    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:22:52.839968 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:52.839423    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:52.839968 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:52.839510    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:22:52.839968 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:52.839917    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:22:53.839275 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:53.839232    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:53.839456 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:53.839357    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:22:54.406040 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:54.406004    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fprgn\" (UniqueName: \"kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn\") pod \"network-check-target-tzv8r\" (UID: \"63b300d3-efba-49ec-8760-7b84c44010d2\") " pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:22:54.406433 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:54.406191    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 22:22:54.406433 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:54.406213    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 22:22:54.406433 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:54.406225    2550 projected.go:196] Error preparing data for projected volume kube-api-access-fprgn for pod openshift-network-diagnostics/network-check-target-tzv8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:22:54.406433 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:54.406283    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn podName:63b300d3-efba-49ec-8760-7b84c44010d2 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:02.406267419 +0000 UTC m=+18.183971449 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fprgn" (UniqueName: "kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn") pod "network-check-target-tzv8r" (UID: "63b300d3-efba-49ec-8760-7b84c44010d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:22:54.506750 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:54.506641    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:54.506750 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:54.506740    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:54.507026 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:54.506789    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:22:54.507026 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:54.506862    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 22:22:54.507026 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:54.506866    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs podName:116caf25-c9e5-47da-a793-bf52c4aa1f5e nodeName:}" failed. No retries permitted until 2026-07-20 22:23:02.506845499 +0000 UTC m=+18.284549533 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs") pod "network-metrics-daemon-m5bvh" (UID: "116caf25-c9e5-47da-a793-bf52c4aa1f5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:22:54.507026 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:54.506922    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret podName:d7df0b06-2bc3-45b8-8443-6c4583b2ca28 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:02.506907184 +0000 UTC m=+18.284611221 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret") pod "global-pull-secret-syncer-89dp7" (UID: "d7df0b06-2bc3-45b8-8443-6c4583b2ca28") : object "kube-system"/"original-pull-secret" not registered
Jul 20 22:22:54.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:54.839758    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:54.840330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:54.839799    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:22:54.840330 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:54.839874    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:22:54.840330 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:54.839959    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:22:55.839209 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:55.839170    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:55.839682 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:55.839313    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:22:56.838861 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:56.838819    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:22:56.838861 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:56.838850    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:56.839115 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:56.838941    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:22:56.839115 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:56.839038    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:22:57.838579 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:57.838551    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:57.838983 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:57.838662    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:22:58.838940 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:58.838905    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:22:58.838940 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:58.838921    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:22:58.839909 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:58.839036    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:22:58.839909 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:58.839153    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:22:59.838944 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:22:59.838908    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:22:59.839377 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:22:59.839047    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:23:00.842168 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:00.842142    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:00.842593 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:00.842242    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:23:00.842593 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:00.842320    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:00.842593 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:00.842410    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:23:01.838770 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:01.838731    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:01.838972 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:01.838859    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:23:02.468620 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:02.468586    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fprgn\" (UniqueName: \"kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn\") pod \"network-check-target-tzv8r\" (UID: \"63b300d3-efba-49ec-8760-7b84c44010d2\") " pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:02.469039 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:02.468743    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 22:23:02.469039 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:02.468766    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 22:23:02.469039 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:02.468780    2550 projected.go:196] Error preparing data for projected volume kube-api-access-fprgn for pod openshift-network-diagnostics/network-check-target-tzv8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:23:02.469039 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:02.468836    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn podName:63b300d3-efba-49ec-8760-7b84c44010d2 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:18.468819834 +0000 UTC m=+34.246523864 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fprgn" (UniqueName: "kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn") pod "network-check-target-tzv8r" (UID: "63b300d3-efba-49ec-8760-7b84c44010d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:23:02.569560 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:02.569459    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:02.569759 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:02.569568    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:02.569759 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:02.569622    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:23:02.569759 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:02.569684    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 22:23:02.569759 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:02.569690    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs podName:116caf25-c9e5-47da-a793-bf52c4aa1f5e nodeName:}" failed. No retries permitted until 2026-07-20 22:23:18.569674825 +0000 UTC m=+34.347378872 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs") pod "network-metrics-daemon-m5bvh" (UID: "116caf25-c9e5-47da-a793-bf52c4aa1f5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:23:02.569942 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:02.569783    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret podName:d7df0b06-2bc3-45b8-8443-6c4583b2ca28 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:18.569764894 +0000 UTC m=+34.347468931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret") pod "global-pull-secret-syncer-89dp7" (UID: "d7df0b06-2bc3-45b8-8443-6c4583b2ca28") : object "kube-system"/"original-pull-secret" not registered
Jul 20 22:23:02.838736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:02.838641    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:02.838736 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:02.838674    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:02.838931 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:02.838784    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:23:02.838931 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:02.838899    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:23:03.838880 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:03.838847    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:03.839280 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:03.838972    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:23:04.842341 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:04.842315    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:04.842761 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:04.842427    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:23:04.842761 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:04.842501    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:04.842761 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:04.842594    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:23:05.839000 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:05.838963    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:05.839165 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:05.839087    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:23:06.838539 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:06.838310    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:06.839557 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:06.838346    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:06.839557 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:06.838640    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:23:06.839557 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:06.838697    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:23:07.012104 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.012027    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vnqk2" event={"ID":"2a64e9d5-1c6a-40e0-9d99-9d3ffabaad74","Type":"ContainerStarted","Data":"6e060b917dbb557509bb412db07a7a246c77e94622edb1252ac9c00af0c18d58"}
Jul 20 22:23:07.013618 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.013590    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xsrgm" event={"ID":"2c4d3266-7aaa-4d42-952c-f98e4e29dec4","Type":"ContainerStarted","Data":"f7f9e77c1877ce6a78becfb94bdc06b835dadd3cee47b4fb51ea3b091942aa38"}
Jul 20 22:23:07.015087 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.015062    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7vxbl" event={"ID":"07ea4c4b-623c-44c9-a630-22f2cf3f91f0","Type":"ContainerStarted","Data":"c8c6cc65474c6b739c289996b2a6b1a250ab23914cc8fd4d96f232bba0961ec0"}
Jul 20 22:23:07.016623 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.016603    2550 generic.go:356] "Generic (PLEG): container finished" podID="e1c875ab-aa59-4235-ac20-201d722fcc7f" containerID="9eb245781f3a6211b7977dcdb41855faf6d7074c4890b421a6da806fe618318f" exitCode=0
Jul 20 22:23:07.016728 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.016672    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g7tg5" event={"ID":"e1c875ab-aa59-4235-ac20-201d722fcc7f","Type":"ContainerDied","Data":"9eb245781f3a6211b7977dcdb41855faf6d7074c4890b421a6da806fe618318f"}
Jul 20 22:23:07.018237 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.018217    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc" event={"ID":"01eff038-0ad7-4c30-b246-52001c256b89","Type":"ContainerStarted","Data":"0dfcfa70f5a833fa621b5d5248a0a815f3ed0376dda05cf4e5d22a4489b62738"}
Jul 20 22:23:07.019632 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.019573    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bw9fm" event={"ID":"b94e391c-6611-4879-a037-77852e20a2af","Type":"ContainerStarted","Data":"08a4d2f480611ff3226f9836eadb797a51f906075b18eebcdb367783c0fce46f"}
Jul 20 22:23:07.020954 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.020935    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-85xd4" event={"ID":"2b9d8d5a-7d89-4556-b963-ef432f0b749e","Type":"ContainerStarted","Data":"fbf4e7f35cf5bc7ceac2ae42c94dd4e6ba27260bfd768de5e2970f0f7457e152"}
Jul 20 22:23:07.028303 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.028266    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-16.ec2.internal" podStartSLOduration=21.02825348 podStartE2EDuration="21.02825348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:22:50.003694082 +0000 UTC m=+5.781398147" watchObservedRunningTime="2026-07-20 22:23:07.02825348 +0000 UTC m=+22.805957531"
Jul 20 22:23:07.042561 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.042523    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vnqk2" podStartSLOduration=4.384340983 podStartE2EDuration="23.042511355s" totalImagesPullingTime="18.658170372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:07.028129923 +0000 UTC m=+22.805833974" watchObservedRunningTime="2026-07-20 22:23:07.042511355 +0000 UTC m=+22.820215460"
Jul 20 22:23:07.058487 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.058447    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bw9fm" podStartSLOduration=3.394664419 podStartE2EDuration="22.058433856s" totalImagesPullingTime="18.663769437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:07.041924658 +0000 UTC m=+22.819628730" watchObservedRunningTime="2026-07-20 22:23:07.058433856 +0000 UTC m=+22.836137908"
Jul 20 22:23:07.058622 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.058604    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xsrgm" podStartSLOduration=4.176999334 podStartE2EDuration="23.058596717s" totalImagesPullingTime="18.881597383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:07.058339591 +0000 UTC m=+22.836043643" watchObservedRunningTime="2026-07-20 22:23:07.058596717 +0000 UTC m=+22.836300768"
Jul 20 22:23:07.076857 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.075747    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7vxbl" podStartSLOduration=4.420455629 podStartE2EDuration="23.075701071s" totalImagesPullingTime="18.655245442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:07.074750563 +0000 UTC m=+22.852454614" watchObservedRunningTime="2026-07-20 22:23:07.075701071 +0000 UTC m=+22.853405126"
Jul 20 22:23:07.094696 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.094659    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-85xd4" podStartSLOduration=4.463894093 podStartE2EDuration="23.094648029s" totalImagesPullingTime="18.630753936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:07.09401709 +0000 UTC m=+22.871721142" watchObservedRunningTime="2026-07-20 22:23:07.094648029 +0000 UTC m=+22.872352080"
Jul 20 22:23:07.207687 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.207658    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:07.288949 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.288889    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:07.838730 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.838502    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:07.839447 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:07.838849    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:23:07.882914 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:07.882876    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:08.017890 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.017856    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-85xd4"
Jul 20 22:23:08.019036 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.019015    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-85xd4"
Jul 20 22:23:08.023931 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.023907    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:08.025154 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.025117    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x8hch" event={"ID":"5d198525-03ef-41a7-80f8-ab1f2fff16ca","Type":"ContainerStarted","Data":"817bdf1042763c4644b82f7e106a62fa1e4653b154dc37dee72bedfb5900b115"}
Jul 20 22:23:08.029086 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.029065    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:23:08.029644 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.029622    2550 generic.go:356] "Generic (PLEG): container finished" podID="87b10474-5ce4-419f-bf63-d9d4b32e7861" containerID="f2ee093f2c4a596f96ad882451b027fbd2e537fc99cbb009d08da39540abdc33" exitCode=1
Jul 20 22:23:08.031103 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.030101    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8985" event={"ID":"87b10474-5ce4-419f-bf63-d9d4b32e7861","Type":"ContainerStarted","Data":"ee3bf5dae0a85b10c2430c7e434aec2590621d671a18f9cff412ef243c898425"}
Jul 20 22:23:08.031103 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.030172    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8985" event={"ID":"87b10474-5ce4-419f-bf63-d9d4b32e7861","Type":"ContainerStarted","Data":"b5d27def332994ce30ca6c5fad0c9fcac7bac2106fbe8cb1cab07bde505aab10"}
Jul 20 22:23:08.031103 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.030223    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8985" event={"ID":"87b10474-5ce4-419f-bf63-d9d4b32e7861","Type":"ContainerStarted","Data":"080e1e4b124834ebc14197861174e769be1c603b6ee13fa6f6fc7a80794a05ea"}
Jul 20 22:23:08.031103 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.030270    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8985" event={"ID":"87b10474-5ce4-419f-bf63-d9d4b32e7861","Type":"ContainerStarted","Data":"a46f295547eb5c11afd0f0b9a446fa44e3e5cec44a3ac5e66e2c817f410b61da"}
Jul 20 22:23:08.031103 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.030316    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8985" event={"ID":"87b10474-5ce4-419f-bf63-d9d4b32e7861","Type":"ContainerDied","Data":"f2ee093f2c4a596f96ad882451b027fbd2e537fc99cbb009d08da39540abdc33"}
Jul 20 22:23:08.031103 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.030365    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8985" event={"ID":"87b10474-5ce4-419f-bf63-d9d4b32e7861","Type":"ContainerStarted","Data":"1f4d88647462b44171b28ed22d15503449646f9fc2e4a5718a2f4473dd0e1bae"}
Jul 20 22:23:08.031462 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.031353    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-85xd4"
Jul 20 22:23:08.031519 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.031511    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-85xd4"
Jul 20 22:23:08.100493 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.100390    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x8hch" podStartSLOduration=5.443002113 podStartE2EDuration="24.100369922s" totalImagesPullingTime="18.657367809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:08.076731146 +0000 UTC m=+23.854435194" watchObservedRunningTime="2026-07-20 22:23:08.100369922 +0000 UTC m=+23.878073976"
Jul 20 22:23:08.131676 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.131641    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:08.226055 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.226020    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:08.339853 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.339826    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:08.407109 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.407087    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:08.441431 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.441396    2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 20 22:23:08.744344 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.744275    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:08.798541 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.798455    2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T22:23:08.441415073Z","UUID":"d69a407b-676c-408b-9101-f78540ccbb32","Handler":null,"Name":"","Endpoint":""}
Jul 20 22:23:08.800388 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.800364    2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 20 22:23:08.800511 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.800402    2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 20 22:23:08.815021 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.814944    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:08.842020 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.841984    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:08.842020 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:08.841996    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:08.842458 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:08.842100    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:23:08.842458 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:08.842232    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:23:09.035032 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:09.034879    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc" event={"ID":"01eff038-0ad7-4c30-b246-52001c256b89","Type":"ContainerStarted","Data":"589dacc712875e75560785150b2991f1f666877cbf2d8eba77cdfcf0583c06d5"}
Jul 20 22:23:09.531780 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:09.531658    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:09.609945 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:09.609893    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:09.838710 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:09.838568    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:09.838824 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:09.838799    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:23:09.960507 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:09.960472    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:10.039733 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:10.039658    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc" event={"ID":"01eff038-0ad7-4c30-b246-52001c256b89","Type":"ContainerStarted","Data":"03380d5da1cccc7659806c29d2bdb91fd7c473fed4bea6237847e806ba06d749"}
Jul 20 22:23:10.058193 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:10.058163    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:10.062969 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:10.062923    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tnzsc" podStartSLOduration=3.643531698 podStartE2EDuration="25.062905117s" totalImagesPullingTime="21.419373419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:10.06238506 +0000 UTC m=+25.840089112" watchObservedRunningTime="2026-07-20 22:23:10.062905117 +0000 UTC m=+25.840609176"
Jul 20 22:23:10.839039 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:10.839011    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:10.839225 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:10.839011    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:10.839225 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:10.839146    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:23:10.839225 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:10.839207    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:23:11.045209 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:11.045183    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:23:11.045697 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:11.045627    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8985" event={"ID":"87b10474-5ce4-419f-bf63-d9d4b32e7861","Type":"ContainerStarted","Data":"fcd751c04d06a56d1abc3fc0d30f0a4b9ac7ca3878e108b7e69dddbb20561e56"}
Jul 20 22:23:11.838872 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:11.838847    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:11.838970 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:11.838949    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:23:12.049981 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:12.049913    2550 generic.go:356] "Generic (PLEG): container finished" podID="e1c875ab-aa59-4235-ac20-201d722fcc7f" containerID="403b531af99bb96f803d339903244b755dcefa528d68c78eddc494ea79f5ea70" exitCode=0
Jul 20 22:23:12.049981 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:12.049950    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g7tg5" event={"ID":"e1c875ab-aa59-4235-ac20-201d722fcc7f","Type":"ContainerDied","Data":"403b531af99bb96f803d339903244b755dcefa528d68c78eddc494ea79f5ea70"}
Jul 20 22:23:12.842466 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:12.842284    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:12.842667 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:12.842328    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:12.842667 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:12.842558    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:23:12.842667 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:12.842655    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:23:13.055102 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:13.055076    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:23:13.055488 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:13.055416    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8985" event={"ID":"87b10474-5ce4-419f-bf63-d9d4b32e7861","Type":"ContainerStarted","Data":"087d7859b52f7f1c150f24a5748c84e45a9e5079e1e665a74f832ac5e8b47b51"}
Jul 20 22:23:13.055658 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:13.055643    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:23:13.055875 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:13.055858    2550 scope.go:117] "RemoveContainer" containerID="f2ee093f2c4a596f96ad882451b027fbd2e537fc99cbb009d08da39540abdc33"
Jul 20 22:23:13.064291 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:13.064271    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:13.071554 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:13.071537    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:23:13.142554 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:13.142533    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:13.838550 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:13.838488    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:13.838676 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:13.838590    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:23:14.059886 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.059855    2550 generic.go:356] "Generic (PLEG): container finished" podID="e1c875ab-aa59-4235-ac20-201d722fcc7f" containerID="c0172ea88d7b0abda2120b521a9cdcd44580845fe8af0eb79436f4bec02de412" exitCode=0
Jul 20 22:23:14.060296 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.059937    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g7tg5" event={"ID":"e1c875ab-aa59-4235-ac20-201d722fcc7f","Type":"ContainerDied","Data":"c0172ea88d7b0abda2120b521a9cdcd44580845fe8af0eb79436f4bec02de412"}
Jul 20 22:23:14.066133 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.066118    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:23:14.066517 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.066493    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8985" event={"ID":"87b10474-5ce4-419f-bf63-d9d4b32e7861","Type":"ContainerStarted","Data":"3ab5817095d2f7c9e5a938c7c8e59da21d42ad20c85499181101178af69b5d00"}
Jul 20 22:23:14.066709 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.066677    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:23:14.066836 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.066797    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:23:14.081140 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.081119    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:23:14.121392 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.121317    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h8985" podStartSLOduration=9.582764012 podStartE2EDuration="29.12130308s" totalImagesPullingTime="19.538539068s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:14.120891427 +0000 UTC m=+29.898595478" watchObservedRunningTime="2026-07-20 22:23:14.12130308 +0000 UTC m=+29.899007132"
Jul 20 22:23:14.548261 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.548228    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-89dp7"]
Jul 20 22:23:14.548443 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.548366    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:14.548487 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:14.548468    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:23:14.552633 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.552611    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tzv8r"]
Jul 20 22:23:14.552796 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.552696    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:14.552868 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:14.552791    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:23:14.553135 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.553118    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m5bvh"]
Jul 20 22:23:14.553217 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:14.553197    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:14.553313 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:14.553296    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:23:15.839227 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:15.839192    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:15.839227 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:15.839219    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:15.839758 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:15.839219    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:15.839758 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:15.839324    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:23:15.839758 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:15.839433    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:23:15.839758 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:15.839507    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:23:16.075328 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:16.075300    2550 generic.go:356] "Generic (PLEG): container finished" podID="e1c875ab-aa59-4235-ac20-201d722fcc7f" containerID="3dc8fc1aec1c31a3a067c3c5b8c1a99917d66cf2cdd84977c9fa63a36a9a9f05" exitCode=0
Jul 20 22:23:16.075470 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:16.075384    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g7tg5" event={"ID":"e1c875ab-aa59-4235-ac20-201d722fcc7f","Type":"ContainerDied","Data":"3dc8fc1aec1c31a3a067c3c5b8c1a99917d66cf2cdd84977c9fa63a36a9a9f05"}
Jul 20 22:23:17.839356 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:17.839324    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:17.839972 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:17.839421    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:17.839972 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:17.839429    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:17.839972 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:17.839421    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:23:17.839972 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:17.839514    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:23:17.839972 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:17.839599    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:23:18.482067 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:18.481995    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fprgn\" (UniqueName: \"kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn\") pod \"network-check-target-tzv8r\" (UID: \"63b300d3-efba-49ec-8760-7b84c44010d2\") " pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:18.482229 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:18.482135    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 22:23:18.482229 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:18.482153    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 22:23:18.482229 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:18.482162    2550 projected.go:196] Error preparing data for projected volume kube-api-access-fprgn for pod openshift-network-diagnostics/network-check-target-tzv8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:23:18.482229 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:18.482217    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn podName:63b300d3-efba-49ec-8760-7b84c44010d2 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:50.482198941 +0000 UTC m=+66.259902994 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fprgn" (UniqueName: "kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn") pod "network-check-target-tzv8r" (UID: "63b300d3-efba-49ec-8760-7b84c44010d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 22:23:18.582560 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:18.582523    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:18.582746 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:18.582581    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:18.582746 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:18.582671    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 22:23:18.582746 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:18.582675    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:23:18.582746 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:18.582731    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret podName:d7df0b06-2bc3-45b8-8443-6c4583b2ca28 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:50.582705036 +0000 UTC m=+66.360409066 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret") pod "global-pull-secret-syncer-89dp7" (UID: "d7df0b06-2bc3-45b8-8443-6c4583b2ca28") : object "kube-system"/"original-pull-secret" not registered
Jul 20 22:23:18.582746 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:18.582749    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs podName:116caf25-c9e5-47da-a793-bf52c4aa1f5e nodeName:}" failed. No retries permitted until 2026-07-20 22:23:50.582739939 +0000 UTC m=+66.360443969 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs") pod "network-metrics-daemon-m5bvh" (UID: "116caf25-c9e5-47da-a793-bf52c4aa1f5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 22:23:19.838696 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:19.838491    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:19.839164 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:19.838549    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:19.839164 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:19.838801    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzv8r" podUID="63b300d3-efba-49ec-8760-7b84c44010d2"
Jul 20 22:23:19.839164 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:19.838889    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m5bvh" podUID="116caf25-c9e5-47da-a793-bf52c4aa1f5e"
Jul 20 22:23:19.839164 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:19.838549    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:19.839164 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:19.838978    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-89dp7" podUID="d7df0b06-2bc3-45b8-8443-6c4583b2ca28"
Jul 20 22:23:20.590194 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.590163    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-16.ec2.internal" event="NodeReady"
Jul 20 22:23:20.590366 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.590312    2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 20 22:23:20.639438 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.639406    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x6w7q"]
Jul 20 22:23:20.671001 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.670971    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-548zv"]
Jul 20 22:23:20.671137 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.671083    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:20.674004 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.673985    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 20 22:23:20.674732 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.674688    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 20 22:23:20.677371 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.674873    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vb99p\""
Jul 20 22:23:20.686880 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.686859    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-548zv"]
Jul 20 22:23:20.686949 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.686894    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x6w7q"]
Jul 20 22:23:20.687001 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.686974    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-548zv"
Jul 20 22:23:20.692817 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.692606    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 20 22:23:20.692817 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.692651    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 20 22:23:20.692817 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.692681    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 20 22:23:20.693047 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.692946    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4d65x\""
Jul 20 22:23:20.796186 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.796151    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5302090-68d3-401b-b54f-dd5c292e7d10-cert\") pod \"ingress-canary-548zv\" (UID: \"c5302090-68d3-401b-b54f-dd5c292e7d10\") " pod="openshift-ingress-canary/ingress-canary-548zv"
Jul 20 22:23:20.796186 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.796188    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwgvt\" (UniqueName: \"kubernetes.io/projected/69c9b463-2c26-4b92-b415-c5bf20afb209-kube-api-access-rwgvt\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:20.796378 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.796232    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:20.796378 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.796252    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c9b463-2c26-4b92-b415-c5bf20afb209-tmp-dir\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:20.796378 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.796276    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m49d\" (UniqueName: \"kubernetes.io/projected/c5302090-68d3-401b-b54f-dd5c292e7d10-kube-api-access-5m49d\") pod \"ingress-canary-548zv\" (UID: \"c5302090-68d3-401b-b54f-dd5c292e7d10\") " pod="openshift-ingress-canary/ingress-canary-548zv"
Jul 20 22:23:20.796378 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.796300    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/69c9b463-2c26-4b92-b415-c5bf20afb209-config-volume\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:20.804894 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.804865    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:20.897355 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.897289    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5302090-68d3-401b-b54f-dd5c292e7d10-cert\") pod \"ingress-canary-548zv\" (UID: \"c5302090-68d3-401b-b54f-dd5c292e7d10\") " pod="openshift-ingress-canary/ingress-canary-548zv"
Jul 20 22:23:20.897355 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.897324    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwgvt\" (UniqueName: \"kubernetes.io/projected/69c9b463-2c26-4b92-b415-c5bf20afb209-kube-api-access-rwgvt\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:20.897909 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.897382    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:20.897909 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.897407    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c9b463-2c26-4b92-b415-c5bf20afb209-tmp-dir\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:20.897909 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.897441    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m49d\" (UniqueName: \"kubernetes.io/projected/c5302090-68d3-401b-b54f-dd5c292e7d10-kube-api-access-5m49d\") pod \"ingress-canary-548zv\" (UID: \"c5302090-68d3-401b-b54f-dd5c292e7d10\") " pod="openshift-ingress-canary/ingress-canary-548zv"
Jul 20 22:23:20.897909 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:20.897508    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 22:23:20.897909 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.897551    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/69c9b463-2c26-4b92-b415-c5bf20afb209-config-volume\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:20.897909 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:20.897572    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls podName:69c9b463-2c26-4b92-b415-c5bf20afb209 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:21.397552223 +0000 UTC m=+37.175256269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls") pod "dns-default-x6w7q" (UID: "69c9b463-2c26-4b92-b415-c5bf20afb209") : secret "dns-default-metrics-tls" not found
Jul 20 22:23:20.897909 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.897776    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c9b463-2c26-4b92-b415-c5bf20afb209-tmp-dir\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:20.898136 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.897987    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/69c9b463-2c26-4b92-b415-c5bf20afb209-config-volume\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:20.901572 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.901547    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5302090-68d3-401b-b54f-dd5c292e7d10-cert\") pod \"ingress-canary-548zv\" (UID: \"c5302090-68d3-401b-b54f-dd5c292e7d10\") " pod="openshift-ingress-canary/ingress-canary-548zv"
Jul 20 22:23:20.901762 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.901744    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:20.908374 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.908351    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwgvt\" (UniqueName: \"kubernetes.io/projected/69c9b463-2c26-4b92-b415-c5bf20afb209-kube-api-access-rwgvt\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:20.908528 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.908513    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m49d\" (UniqueName: \"kubernetes.io/projected/c5302090-68d3-401b-b54f-dd5c292e7d10-kube-api-access-5m49d\") pod \"ingress-canary-548zv\" (UID: \"c5302090-68d3-401b-b54f-dd5c292e7d10\") " pod="openshift-ingress-canary/ingress-canary-548zv"
Jul 20 22:23:20.996459 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:20.996435    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-548zv"
Jul 20 22:23:21.321363 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:21.321333    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-548zv"]
Jul 20 22:23:21.325468 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:23:21.325440    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5302090_68d3_401b_b54f_dd5c292e7d10.slice/crio-9f76f3a9c8c16fdd1f3e9e58685aab58f2f204bd661a7d182fbb2cb4ac670e5d WatchSource:0}: Error finding container 9f76f3a9c8c16fdd1f3e9e58685aab58f2f204bd661a7d182fbb2cb4ac670e5d: Status 404 returned error can't find the container with id 9f76f3a9c8c16fdd1f3e9e58685aab58f2f204bd661a7d182fbb2cb4ac670e5d
Jul 20 22:23:21.402148 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:21.402095    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:21.402225 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:21.402213    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 22:23:21.402274 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:21.402265    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls podName:69c9b463-2c26-4b92-b415-c5bf20afb209 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:22.402250198 +0000 UTC m=+38.179954228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls") pod "dns-default-x6w7q" (UID: "69c9b463-2c26-4b92-b415-c5bf20afb209") : secret "dns-default-metrics-tls" not found
Jul 20 22:23:21.838854 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:21.838817    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:21.839035 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:21.838817    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:21.839035 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:21.838817    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:21.842978 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:21.842957    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 22:23:21.843091 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:21.843011    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wpkcl\""
Jul 20 22:23:21.844045 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:21.843905    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 22:23:21.844045 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:21.843932    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 22:23:21.844045 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:21.843941    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vv84p\""
Jul 20 22:23:21.844045 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:21.843956    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 22:23:22.093601 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:22.093364    2550 generic.go:356] "Generic (PLEG): container finished" podID="e1c875ab-aa59-4235-ac20-201d722fcc7f" containerID="6f61edd656f3abced98d6a1a83c200ea4dc6879929fdc02311b3b3b65d39f902" exitCode=0
Jul 20 22:23:22.093601 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:22.093451    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g7tg5" event={"ID":"e1c875ab-aa59-4235-ac20-201d722fcc7f","Type":"ContainerDied","Data":"6f61edd656f3abced98d6a1a83c200ea4dc6879929fdc02311b3b3b65d39f902"}
Jul 20 22:23:22.095538 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:22.095508    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-548zv" event={"ID":"c5302090-68d3-401b-b54f-dd5c292e7d10","Type":"ContainerStarted","Data":"9f76f3a9c8c16fdd1f3e9e58685aab58f2f204bd661a7d182fbb2cb4ac670e5d"}
Jul 20 22:23:22.409990 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:22.409906    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:22.410136 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:22.410076    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 22:23:22.410178 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:22.410151    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls podName:69c9b463-2c26-4b92-b415-c5bf20afb209 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:24.410131164 +0000 UTC m=+40.187835197 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls") pod "dns-default-x6w7q" (UID: "69c9b463-2c26-4b92-b415-c5bf20afb209") : secret "dns-default-metrics-tls" not found
Jul 20 22:23:23.101834 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:23.101798    2550 generic.go:356] "Generic (PLEG): container finished" podID="e1c875ab-aa59-4235-ac20-201d722fcc7f" containerID="6e21ec50ade72af51593303dfbd9acf5718981d7798f417aeed9dfc943563ef1" exitCode=0
Jul 20 22:23:23.102327 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:23.101848    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g7tg5" event={"ID":"e1c875ab-aa59-4235-ac20-201d722fcc7f","Type":"ContainerDied","Data":"6e21ec50ade72af51593303dfbd9acf5718981d7798f417aeed9dfc943563ef1"}
Jul 20 22:23:24.108284 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:24.108205    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g7tg5" event={"ID":"e1c875ab-aa59-4235-ac20-201d722fcc7f","Type":"ContainerStarted","Data":"0f363c3bf51ce1a0dd42a3edc2af2cbb0fb747054fd3cd20736d348967564ed8"}
Jul 20 22:23:24.109779 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:24.109753    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-548zv" event={"ID":"c5302090-68d3-401b-b54f-dd5c292e7d10","Type":"ContainerStarted","Data":"b9d03dd75267973a2078ba61ce6c33e35474276153d4ef425f20ddb32a77c87f"}
Jul 20 22:23:24.134327 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:24.134275    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-g7tg5" podStartSLOduration=9.481519494 podStartE2EDuration="40.134259046s" totalImagesPullingTime="30.652739552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:24.133113422 +0000 UTC m=+39.910817473" watchObservedRunningTime="2026-07-20 22:23:24.134259046 +0000 UTC m=+39.911963097"
Jul 20 22:23:24.147407 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:24.147364    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:24.152922 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:24.152878    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-548zv" podStartSLOduration=1.746098102 podStartE2EDuration="4.152863018s" totalImagesPullingTime="2.406764916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:24.152447399 +0000 UTC m=+39.930151462" watchObservedRunningTime="2026-07-20 22:23:24.152863018 +0000 UTC m=+39.930567070"
Jul 20 22:23:24.267467 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:24.267427    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:24.369092 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:24.369004    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:24.427158 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:24.427116    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:24.427308 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:24.427269    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 22:23:24.427344 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:24.427332    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls podName:69c9b463-2c26-4b92-b415-c5bf20afb209 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:28.42731651 +0000 UTC m=+44.205020540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls") pod "dns-default-x6w7q" (UID: "69c9b463-2c26-4b92-b415-c5bf20afb209") : secret "dns-default-metrics-tls" not found
Jul 20 22:23:26.612985 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:26.612951    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:26.713670 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:26.713641    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:27.612057 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:27.612018    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:27.712026 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:27.711981    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:27.816305 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:27.816275    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:28.019894 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:28.019858    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:28.105915 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:28.105878    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:28.453409 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:28.453381    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:28.453552 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:28.453531    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 22:23:28.453624 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:28.453614    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls podName:69c9b463-2c26-4b92-b415-c5bf20afb209 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:36.453596527 +0000 UTC m=+52.231300574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls") pod "dns-default-x6w7q" (UID: "69c9b463-2c26-4b92-b415-c5bf20afb209") : secret "dns-default-metrics-tls" not found
Jul 20 22:23:33.640337 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.640296    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-jtsls"]
Jul 20 22:23:33.683833 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.683795    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"]
Jul 20 22:23:33.683994 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.683916    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jtsls"
Jul 20 22:23:33.686311 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.686290    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-c29qp\""
Jul 20 22:23:33.697544 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.697523    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"]
Jul 20 22:23:33.697659 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.697642    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"
Jul 20 22:23:33.700488 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.700465    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 20 22:23:33.700587 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.700472    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-dqx5t\""
Jul 20 22:23:33.700683 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.700668    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 20 22:23:33.700882 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.700870    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 20 22:23:33.701199 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.701187    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 20 22:23:33.715368 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.715346    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7zqcq"]
Jul 20 22:23:33.715470 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.715453    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"
Jul 20 22:23:33.717839 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.717823    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 20 22:23:33.717939 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.717927    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 20 22:23:33.718002 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.717934    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f82s7\""
Jul 20 22:23:33.718055 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.718038    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 20 22:23:33.741109 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.741088    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-jtsls"]
Jul 20 22:23:33.741236 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.741126    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"]
Jul 20 22:23:33.741236 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.741144    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7zqcq"]
Jul 20 22:23:33.741236 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.741158    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"]
Jul 20 22:23:33.741236 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.741211    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7zqcq"
Jul 20 22:23:33.743627 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.743608    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 20 22:23:33.743829 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.743815    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 20 22:23:33.743921 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.743910    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zjck8\""
Jul 20 22:23:33.745248 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.745227    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"]
Jul 20 22:23:33.761124 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.761103    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"]
Jul 20 22:23:33.761230 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.761218    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:33.765279 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.765256    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 20 22:23:33.765620 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.765604    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-t9kv4\""
Jul 20 22:23:33.766227 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.766209    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 20 22:23:33.768475 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.768453    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 20 22:23:33.773599 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.773573    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 20 22:23:33.775221 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.775200    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-54dbf84666-xw84d"]
Jul 20 22:23:33.775330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.775318    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"
Jul 20 22:23:33.777615 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.777597    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 20 22:23:33.778190 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.778174    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-2p979\""
Jul 20 22:23:33.778414 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.778399    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 20 22:23:33.778474 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.778451    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 20 22:23:33.778556 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.778538    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 20 22:23:33.789375 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.789355    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9vbh\" (UniqueName: \"kubernetes.io/projected/8b8b3234-3157-4414-8b71-0c7eacd25f02-kube-api-access-c9vbh\") pod \"network-check-source-7bc5b6bf85-jtsls\" (UID: \"8b8b3234-3157-4414-8b71-0c7eacd25f02\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jtsls"
Jul 20 22:23:33.789464 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.789390    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f87hx\" (UniqueName: \"kubernetes.io/projected/22aad18a-8d9e-4509-9773-5906515b3782-kube-api-access-f87hx\") pod \"kube-storage-version-migrator-operator-7658d9b749-87h8h\" (UID: \"22aad18a-8d9e-4509-9773-5906515b3782\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"
Jul 20 22:23:33.789464 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.789409    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22aad18a-8d9e-4509-9773-5906515b3782-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-87h8h\" (UID: \"22aad18a-8d9e-4509-9773-5906515b3782\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"
Jul 20 22:23:33.789535 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.789501    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22aad18a-8d9e-4509-9773-5906515b3782-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-87h8h\" (UID: \"22aad18a-8d9e-4509-9773-5906515b3782\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"
Jul 20 22:23:33.791370 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.791348    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"]
Jul 20 22:23:33.791469 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.791379    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"]
Jul 20 22:23:33.791469 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.791394    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54dbf84666-xw84d"]
Jul 20 22:23:33.791469 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.791459    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:33.792795 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.792777    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f4fc9474c-6wjcg"]
Jul 20 22:23:33.794503 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.794490    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 20 22:23:33.794769 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.794754    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 20 22:23:33.795037 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.795020    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 20 22:23:33.795289 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.795274    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 20 22:23:33.795328 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.795311    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 20 22:23:33.795447 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.795432    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-lfzmn\""
Jul 20 22:23:33.795499 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.795484    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 20 22:23:33.805402 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.805385    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.807683 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.807665    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 20 22:23:33.807922 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.807909    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 20 22:23:33.808186 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.808167    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-42787\""
Jul 20 22:23:33.808267 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.808199    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 20 22:23:33.813656 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.813634    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f4fc9474c-6wjcg"]
Jul 20 22:23:33.814065 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.814047    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 20 22:23:33.890142 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890114    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zvqr\" (UniqueName: \"kubernetes.io/projected/2e863cc6-8c15-4725-a3b5-9748719edb3c-kube-api-access-9zvqr\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:33.890142 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890141    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6mrqc\" (UID: \"65f38e33-fccd-4881-a0e1-466815c69780\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"
Jul 20 22:23:33.890298 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890157    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21d248fd-bc94-4517-86e4-79e38827640e-config\") pod \"service-ca-operator-5bcbc94c78-9kpkv\" (UID: \"21d248fd-bc94-4517-86e4-79e38827640e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"
Jul 20 22:23:33.890298 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890178    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f5hv\" (UniqueName: \"kubernetes.io/projected/65f38e33-fccd-4881-a0e1-466815c69780-kube-api-access-5f5hv\") pod \"cluster-samples-operator-79b7c55f87-6mrqc\" (UID: \"65f38e33-fccd-4881-a0e1-466815c69780\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"
Jul 20 22:23:33.890298 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890205    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4210e665-12b0-46da-926b-0fa1822f9b63-image-registry-private-configuration\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.890298 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890230    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4210e665-12b0-46da-926b-0fa1822f9b63-ca-trust-extracted\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.890468 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890306    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndv4j\" (UniqueName: \"kubernetes.io/projected/e9e5fca5-4619-48ab-b7a7-51ac2743a899-kube-api-access-ndv4j\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:33.890468 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890329    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzcdw\" (UniqueName: \"kubernetes.io/projected/db2efb42-daa5-4924-8c76-47f89a07bc7d-kube-api-access-rzcdw\") pod \"volume-data-source-validator-78cf4c6f6-7zqcq\" (UID: \"db2efb42-daa5-4924-8c76-47f89a07bc7d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7zqcq"
Jul 20 22:23:33.890468 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890349    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpkrx\" (UniqueName: \"kubernetes.io/projected/21d248fd-bc94-4517-86e4-79e38827640e-kube-api-access-bpkrx\") pod \"service-ca-operator-5bcbc94c78-9kpkv\" (UID: \"21d248fd-bc94-4517-86e4-79e38827640e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"
Jul 20 22:23:33.890468 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890365    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-bound-sa-token\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.890468 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890424    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9vbh\" (UniqueName: \"kubernetes.io/projected/8b8b3234-3157-4414-8b71-0c7eacd25f02-kube-api-access-c9vbh\") pod \"network-check-source-7bc5b6bf85-jtsls\" (UID: \"8b8b3234-3157-4414-8b71-0c7eacd25f02\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jtsls"
Jul 20 22:23:33.890681 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890492    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f87hx\" (UniqueName: \"kubernetes.io/projected/22aad18a-8d9e-4509-9773-5906515b3782-kube-api-access-f87hx\") pod \"kube-storage-version-migrator-operator-7658d9b749-87h8h\" (UID: \"22aad18a-8d9e-4509-9773-5906515b3782\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"
Jul 20 22:23:33.890681 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890511    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:33.890681 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890529    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21d248fd-bc94-4517-86e4-79e38827640e-serving-cert\") pod \"service-ca-operator-5bcbc94c78-9kpkv\" (UID: \"21d248fd-bc94-4517-86e4-79e38827640e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"
Jul 20 22:23:33.890681 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890552    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22aad18a-8d9e-4509-9773-5906515b3782-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-87h8h\" (UID: \"22aad18a-8d9e-4509-9773-5906515b3782\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"
Jul 20 22:23:33.890681 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890579    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-default-certificate\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:33.890681 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890605    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:33.890681 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890625    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4210e665-12b0-46da-926b-0fa1822f9b63-registry-certificates\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.890681 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890660    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-stats-auth\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:33.890681 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890679    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e9e5fca5-4619-48ab-b7a7-51ac2743a899-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:33.891045 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890696    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4210e665-12b0-46da-926b-0fa1822f9b63-trusted-ca\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.891045 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890732    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:33.891045 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890779    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22aad18a-8d9e-4509-9773-5906515b3782-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-87h8h\" (UID: \"22aad18a-8d9e-4509-9773-5906515b3782\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"
Jul 20 22:23:33.891045 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890807    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4210e665-12b0-46da-926b-0fa1822f9b63-installation-pull-secrets\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.891045 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890835    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6skj\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-kube-api-access-r6skj\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.891045 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.890850    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.891218 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.891047    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22aad18a-8d9e-4509-9773-5906515b3782-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-87h8h\" (UID: \"22aad18a-8d9e-4509-9773-5906515b3782\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"
Jul 20 22:23:33.894093 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.894074    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22aad18a-8d9e-4509-9773-5906515b3782-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-87h8h\" (UID: \"22aad18a-8d9e-4509-9773-5906515b3782\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"
Jul 20 22:23:33.901821 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.901796    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f87hx\" (UniqueName: \"kubernetes.io/projected/22aad18a-8d9e-4509-9773-5906515b3782-kube-api-access-f87hx\") pod \"kube-storage-version-migrator-operator-7658d9b749-87h8h\" (UID: \"22aad18a-8d9e-4509-9773-5906515b3782\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"
Jul 20 22:23:33.902241 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.902225    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9vbh\" (UniqueName: \"kubernetes.io/projected/8b8b3234-3157-4414-8b71-0c7eacd25f02-kube-api-access-c9vbh\") pod \"network-check-source-7bc5b6bf85-jtsls\" (UID: \"8b8b3234-3157-4414-8b71-0c7eacd25f02\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jtsls"
Jul 20 22:23:33.976088 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.976064    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:33.991892 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.991874    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jtsls"
Jul 20 22:23:33.992068 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992046    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:33.992177 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992095    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4210e665-12b0-46da-926b-0fa1822f9b63-installation-pull-secrets\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.992177 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992140    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6skj\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-kube-api-access-r6skj\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.992177 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992168    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.992282 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:33.992202    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle podName:2e863cc6-8c15-4725-a3b5-9748719edb3c nodeName:}" failed. No retries permitted until 2026-07-20 22:23:34.492184448 +0000 UTC m=+50.269888499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle") pod "router-default-54dbf84666-xw84d" (UID: "2e863cc6-8c15-4725-a3b5-9748719edb3c") : configmap references non-existent config key: service-ca.crt
Jul 20 22:23:33.992282 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992220    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zvqr\" (UniqueName: \"kubernetes.io/projected/2e863cc6-8c15-4725-a3b5-9748719edb3c-kube-api-access-9zvqr\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:33.992282 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992240    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6mrqc\" (UID: \"65f38e33-fccd-4881-a0e1-466815c69780\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"
Jul 20 22:23:33.992282 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992259    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21d248fd-bc94-4517-86e4-79e38827640e-config\") pod \"service-ca-operator-5bcbc94c78-9kpkv\" (UID: \"21d248fd-bc94-4517-86e4-79e38827640e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"
Jul 20 22:23:33.992480 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992298    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f5hv\" (UniqueName: \"kubernetes.io/projected/65f38e33-fccd-4881-a0e1-466815c69780-kube-api-access-5f5hv\") pod \"cluster-samples-operator-79b7c55f87-6mrqc\" (UID: \"65f38e33-fccd-4881-a0e1-466815c69780\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"
Jul 20 22:23:33.992480 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:33.992322    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 22:23:33.992480 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992329    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4210e665-12b0-46da-926b-0fa1822f9b63-image-registry-private-configuration\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.992480 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:33.992335    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f4fc9474c-6wjcg: secret "image-registry-tls" not found
Jul 20 22:23:33.992480 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992354    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4210e665-12b0-46da-926b-0fa1822f9b63-ca-trust-extracted\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.992480 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:33.992371    2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 20 22:23:33.992480 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992390    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndv4j\" (UniqueName: \"kubernetes.io/projected/e9e5fca5-4619-48ab-b7a7-51ac2743a899-kube-api-access-ndv4j\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:33.992480 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992420    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzcdw\" (UniqueName: \"kubernetes.io/projected/db2efb42-daa5-4924-8c76-47f89a07bc7d-kube-api-access-rzcdw\") pod \"volume-data-source-validator-78cf4c6f6-7zqcq\" (UID: \"db2efb42-daa5-4924-8c76-47f89a07bc7d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7zqcq"
Jul 20 22:23:33.992480 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:33.992447    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls podName:65f38e33-fccd-4881-a0e1-466815c69780 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:34.492426649 +0000 UTC m=+50.270130685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6mrqc" (UID: "65f38e33-fccd-4881-a0e1-466815c69780") : secret "samples-operator-tls" not found
Jul 20 22:23:33.992480 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:33.992482    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls podName:4210e665-12b0-46da-926b-0fa1822f9b63 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:34.492471228 +0000 UTC m=+50.270175258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls") pod "image-registry-6f4fc9474c-6wjcg" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63") : secret "image-registry-tls" not found
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992521    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpkrx\" (UniqueName: \"kubernetes.io/projected/21d248fd-bc94-4517-86e4-79e38827640e-kube-api-access-bpkrx\") pod \"service-ca-operator-5bcbc94c78-9kpkv\" (UID: \"21d248fd-bc94-4517-86e4-79e38827640e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992563    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-bound-sa-token\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992621    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992644    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21d248fd-bc94-4517-86e4-79e38827640e-serving-cert\") pod \"service-ca-operator-5bcbc94c78-9kpkv\" (UID: \"21d248fd-bc94-4517-86e4-79e38827640e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992674    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-default-certificate\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992703    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992751    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4210e665-12b0-46da-926b-0fa1822f9b63-registry-certificates\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992779    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-stats-auth\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992789    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4210e665-12b0-46da-926b-0fa1822f9b63-ca-trust-extracted\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992808    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e9e5fca5-4619-48ab-b7a7-51ac2743a899-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992835    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4210e665-12b0-46da-926b-0fa1822f9b63-trusted-ca\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:33.992899    2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:33.992949    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls podName:e9e5fca5-4619-48ab-b7a7-51ac2743a899 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:34.4929318 +0000 UTC m=+50.270635831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9mdsd" (UID: "e9e5fca5-4619-48ab-b7a7-51ac2743a899") : secret "cluster-monitoring-operator-tls" not found
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.992957    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21d248fd-bc94-4517-86e4-79e38827640e-config\") pod \"service-ca-operator-5bcbc94c78-9kpkv\" (UID: \"21d248fd-bc94-4517-86e4-79e38827640e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"
Jul 20 22:23:33.992991 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:33.992996    2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 20 22:23:33.993704 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:33.993059    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs podName:2e863cc6-8c15-4725-a3b5-9748719edb3c nodeName:}" failed. No retries permitted until 2026-07-20 22:23:34.493043084 +0000 UTC m=+50.270747113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs") pod "router-default-54dbf84666-xw84d" (UID: "2e863cc6-8c15-4725-a3b5-9748719edb3c") : secret "router-metrics-certs-default" not found
Jul 20 22:23:33.993704 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.993528    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4210e665-12b0-46da-926b-0fa1822f9b63-registry-certificates\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.993844 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.993792    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e9e5fca5-4619-48ab-b7a7-51ac2743a899-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:33.993844 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.993817    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4210e665-12b0-46da-926b-0fa1822f9b63-trusted-ca\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.994995 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.994973    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4210e665-12b0-46da-926b-0fa1822f9b63-installation-pull-secrets\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.995086 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.994982    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4210e665-12b0-46da-926b-0fa1822f9b63-image-registry-private-configuration\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:33.995453 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.995433    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21d248fd-bc94-4517-86e4-79e38827640e-serving-cert\") pod \"service-ca-operator-5bcbc94c78-9kpkv\" (UID: \"21d248fd-bc94-4517-86e4-79e38827640e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"
Jul 20 22:23:33.995609 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.995593    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-stats-auth\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:33.995912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:33.995894    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-default-certificate\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:34.004049 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.003960    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpkrx\" (UniqueName: \"kubernetes.io/projected/21d248fd-bc94-4517-86e4-79e38827640e-kube-api-access-bpkrx\") pod \"service-ca-operator-5bcbc94c78-9kpkv\" (UID: \"21d248fd-bc94-4517-86e4-79e38827640e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"
Jul 20 22:23:34.004049 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.003987    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndv4j\" (UniqueName: \"kubernetes.io/projected/e9e5fca5-4619-48ab-b7a7-51ac2743a899-kube-api-access-ndv4j\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:34.004545 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.004521    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzcdw\" (UniqueName: \"kubernetes.io/projected/db2efb42-daa5-4924-8c76-47f89a07bc7d-kube-api-access-rzcdw\") pod \"volume-data-source-validator-78cf4c6f6-7zqcq\" (UID: \"db2efb42-daa5-4924-8c76-47f89a07bc7d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7zqcq"
Jul 20 22:23:34.004625 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.004601    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-bound-sa-token\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:34.005263 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.005239    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"
Jul 20 22:23:34.005597 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.005574    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f5hv\" (UniqueName: \"kubernetes.io/projected/65f38e33-fccd-4881-a0e1-466815c69780-kube-api-access-5f5hv\") pod \"cluster-samples-operator-79b7c55f87-6mrqc\" (UID: \"65f38e33-fccd-4881-a0e1-466815c69780\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"
Jul 20 22:23:34.005864 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.005849    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6skj\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-kube-api-access-r6skj\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:34.006011 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.005996    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zvqr\" (UniqueName: \"kubernetes.io/projected/2e863cc6-8c15-4725-a3b5-9748719edb3c-kube-api-access-9zvqr\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:34.049885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.049857    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7zqcq"
Jul 20 22:23:34.074081 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.074010    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:34.083078 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.083045    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"
Jul 20 22:23:34.182708 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.182512    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-jtsls"]
Jul 20 22:23:34.205372 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.205348    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h"]
Jul 20 22:23:34.207193 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:23:34.207166    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22aad18a_8d9e_4509_9773_5906515b3782.slice/crio-00be099e3e2bf2a50a2a4eda5e72e97ac05b9464cd9c46199fbd7ae1c6fcb894 WatchSource:0}: Error finding container 00be099e3e2bf2a50a2a4eda5e72e97ac05b9464cd9c46199fbd7ae1c6fcb894: Status 404 returned error can't find the container with id 00be099e3e2bf2a50a2a4eda5e72e97ac05b9464cd9c46199fbd7ae1c6fcb894
Jul 20 22:23:34.259033 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.258966    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7zqcq"]
Jul 20 22:23:34.262235 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:23:34.262201    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb2efb42_daa5_4924_8c76_47f89a07bc7d.slice/crio-29b64df0898f343feaaba8adc591da7a26789a862917d831a47fe5b9935d1e69 WatchSource:0}: Error finding container 29b64df0898f343feaaba8adc591da7a26789a862917d831a47fe5b9935d1e69: Status 404 returned error can't find the container with id 29b64df0898f343feaaba8adc591da7a26789a862917d831a47fe5b9935d1e69
Jul 20 22:23:34.278623 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.278599    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv"]
Jul 20 22:23:34.281691 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:23:34.281665    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d248fd_bc94_4517_86e4_79e38827640e.slice/crio-fc8020a90e349b88e0c1dba734a632890d2773290914af9b532982c3c2f9c7bd WatchSource:0}: Error finding container fc8020a90e349b88e0c1dba734a632890d2773290914af9b532982c3c2f9c7bd: Status 404 returned error can't find the container with id fc8020a90e349b88e0c1dba734a632890d2773290914af9b532982c3c2f9c7bd
Jul 20 22:23:34.498890 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.498861    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6mrqc\" (UID: \"65f38e33-fccd-4881-a0e1-466815c69780\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"
Jul 20 22:23:34.499094 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.498927    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:34.499094 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.498952    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:34.499094 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.498977    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:34.499094 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:34.499013    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:34.499094 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:34.499040    2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 20 22:23:34.499094 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:34.499082    2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 20 22:23:34.499322 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:34.499101    2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 20 22:23:34.499322 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:34.499089    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 22:23:34.499322 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:34.499120    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls podName:65f38e33-fccd-4881-a0e1-466815c69780 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:35.499098685 +0000 UTC m=+51.276802737 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6mrqc" (UID: "65f38e33-fccd-4881-a0e1-466815c69780") : secret "samples-operator-tls" not found
Jul 20 22:23:34.499322 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:34.499128    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f4fc9474c-6wjcg: secret "image-registry-tls" not found
Jul 20 22:23:34.499322 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:34.499141    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle podName:2e863cc6-8c15-4725-a3b5-9748719edb3c nodeName:}" failed. No retries permitted until 2026-07-20 22:23:35.499131248 +0000 UTC m=+51.276835283 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle") pod "router-default-54dbf84666-xw84d" (UID: "2e863cc6-8c15-4725-a3b5-9748719edb3c") : configmap references non-existent config key: service-ca.crt
Jul 20 22:23:34.499322 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:34.499158    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs podName:2e863cc6-8c15-4725-a3b5-9748719edb3c nodeName:}" failed. No retries permitted until 2026-07-20 22:23:35.499150108 +0000 UTC m=+51.276854144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs") pod "router-default-54dbf84666-xw84d" (UID: "2e863cc6-8c15-4725-a3b5-9748719edb3c") : secret "router-metrics-certs-default" not found
Jul 20 22:23:34.499322 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:34.499173    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls podName:e9e5fca5-4619-48ab-b7a7-51ac2743a899 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:35.4991648 +0000 UTC m=+51.276868833 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9mdsd" (UID: "e9e5fca5-4619-48ab-b7a7-51ac2743a899") : secret "cluster-monitoring-operator-tls" not found
Jul 20 22:23:34.499322 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:34.499187    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls podName:4210e665-12b0-46da-926b-0fa1822f9b63 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:35.499179268 +0000 UTC m=+51.276883302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls") pod "image-registry-6f4fc9474c-6wjcg" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63") : secret "image-registry-tls" not found
Jul 20 22:23:35.143883 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:35.143840    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h" event={"ID":"22aad18a-8d9e-4509-9773-5906515b3782","Type":"ContainerStarted","Data":"00be099e3e2bf2a50a2a4eda5e72e97ac05b9464cd9c46199fbd7ae1c6fcb894"}
Jul 20 22:23:35.145701 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:35.145666    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv" event={"ID":"21d248fd-bc94-4517-86e4-79e38827640e","Type":"ContainerStarted","Data":"fc8020a90e349b88e0c1dba734a632890d2773290914af9b532982c3c2f9c7bd"}
Jul 20 22:23:35.147120 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:35.147091    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7zqcq" event={"ID":"db2efb42-daa5-4924-8c76-47f89a07bc7d","Type":"ContainerStarted","Data":"29b64df0898f343feaaba8adc591da7a26789a862917d831a47fe5b9935d1e69"}
Jul 20 22:23:35.149615 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:35.149533    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jtsls" event={"ID":"8b8b3234-3157-4414-8b71-0c7eacd25f02","Type":"ContainerStarted","Data":"49e7a9bb32cb83139c2bd76b387c6abaf595ed0943908abcd2fe0025882d312b"}
Jul 20 22:23:35.509056 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:35.508973    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:35.509249 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:35.509078    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6mrqc\" (UID: \"65f38e33-fccd-4881-a0e1-466815c69780\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"
Jul 20 22:23:35.509249 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:35.509086    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 22:23:35.509249 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:35.509105    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f4fc9474c-6wjcg: secret "image-registry-tls" not found
Jul 20 22:23:35.509249 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:35.509165    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls podName:4210e665-12b0-46da-926b-0fa1822f9b63 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:37.509144431 +0000 UTC m=+53.286848462 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls") pod "image-registry-6f4fc9474c-6wjcg" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63") : secret "image-registry-tls" not found
Jul 20 22:23:35.509249 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:35.509208    2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 20 22:23:35.509249 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:35.509237    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:35.509249 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:35.509246    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls podName:65f38e33-fccd-4881-a0e1-466815c69780 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:37.509234133 +0000 UTC m=+53.286938176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6mrqc" (UID: "65f38e33-fccd-4881-a0e1-466815c69780") : secret "samples-operator-tls" not found
Jul 20 22:23:35.509658 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:35.509293    2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 20 22:23:35.509658 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:35.509301    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:35.509658 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:35.509323    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs podName:2e863cc6-8c15-4725-a3b5-9748719edb3c nodeName:}" failed. No retries permitted until 2026-07-20 22:23:37.509313366 +0000 UTC m=+53.287017397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs") pod "router-default-54dbf84666-xw84d" (UID: "2e863cc6-8c15-4725-a3b5-9748719edb3c") : secret "router-metrics-certs-default" not found
Jul 20 22:23:35.509658 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:35.509356    2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 20 22:23:35.509658 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:35.509362    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:35.509658 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:35.509388    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls podName:e9e5fca5-4619-48ab-b7a7-51ac2743a899 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:37.509378168 +0000 UTC m=+53.287082211 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9mdsd" (UID: "e9e5fca5-4619-48ab-b7a7-51ac2743a899") : secret "cluster-monitoring-operator-tls" not found
Jul 20 22:23:35.509658 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:35.509433    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle podName:2e863cc6-8c15-4725-a3b5-9748719edb3c nodeName:}" failed. No retries permitted until 2026-07-20 22:23:37.509424751 +0000 UTC m=+53.287128781 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle") pod "router-default-54dbf84666-xw84d" (UID: "2e863cc6-8c15-4725-a3b5-9748719edb3c") : configmap references non-existent config key: service-ca.crt
Jul 20 22:23:36.519044 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:36.519006    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:36.519418 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:36.519141    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 22:23:36.519418 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:36.519193    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls podName:69c9b463-2c26-4b92-b415-c5bf20afb209 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:52.51917876 +0000 UTC m=+68.296882789 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls") pod "dns-default-x6w7q" (UID: "69c9b463-2c26-4b92-b415-c5bf20afb209") : secret "dns-default-metrics-tls" not found
Jul 20 22:23:37.526974 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:37.526927    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:37.527031    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:37.527065    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6mrqc\" (UID: \"65f38e33-fccd-4881-a0e1-466815c69780\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:37.527130    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle podName:2e863cc6-8c15-4725-a3b5-9748719edb3c nodeName:}" failed. No retries permitted until 2026-07-20 22:23:41.527105156 +0000 UTC m=+57.304809194 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle") pod "router-default-54dbf84666-xw84d" (UID: "2e863cc6-8c15-4725-a3b5-9748719edb3c") : configmap references non-existent config key: service-ca.crt
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:37.527186    2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:37.527197    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:37.527212    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f4fc9474c-6wjcg: secret "image-registry-tls" not found
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:37.527221    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:37.527238    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls podName:65f38e33-fccd-4881-a0e1-466815c69780 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:41.527223273 +0000 UTC m=+57.304927305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6mrqc" (UID: "65f38e33-fccd-4881-a0e1-466815c69780") : secret "samples-operator-tls" not found
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:37.527259    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls podName:4210e665-12b0-46da-926b-0fa1822f9b63 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:41.527249401 +0000 UTC m=+57.304953438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls") pod "image-registry-6f4fc9474c-6wjcg" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63") : secret "image-registry-tls" not found
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:37.527295    2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:37.527331    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs podName:2e863cc6-8c15-4725-a3b5-9748719edb3c nodeName:}" failed. No retries permitted until 2026-07-20 22:23:41.527318788 +0000 UTC m=+57.305022824 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs") pod "router-default-54dbf84666-xw84d" (UID: "2e863cc6-8c15-4725-a3b5-9748719edb3c") : secret "router-metrics-certs-default" not found
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:37.527293    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:37.527330    2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 20 22:23:37.527434 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:37.527423    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls podName:e9e5fca5-4619-48ab-b7a7-51ac2743a899 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:41.527404085 +0000 UTC m=+57.305108132 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9mdsd" (UID: "e9e5fca5-4619-48ab-b7a7-51ac2743a899") : secret "cluster-monitoring-operator-tls" not found
Jul 20 22:23:39.163107 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:39.163005    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jtsls" event={"ID":"8b8b3234-3157-4414-8b71-0c7eacd25f02","Type":"ContainerStarted","Data":"ff4ab0358cf68e36a88fd42b91a07b7db0fd154366c61d8ca814aef3572ba074"}
Jul 20 22:23:39.164577 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:39.164547    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h" event={"ID":"22aad18a-8d9e-4509-9773-5906515b3782","Type":"ContainerStarted","Data":"bf1b0a327bf4a21b9d84d1df92218e87ab9927349ed9c3cce70bf58bd912f0c3"}
Jul 20 22:23:39.166042 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:39.166011    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv" event={"ID":"21d248fd-bc94-4517-86e4-79e38827640e","Type":"ContainerStarted","Data":"15674c5073c1e70e25a6bde9dc491a8154b8fa01ff510e03078ab427960ac017"}
Jul 20 22:23:39.167684 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:39.167666    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7zqcq" event={"ID":"db2efb42-daa5-4924-8c76-47f89a07bc7d","Type":"ContainerStarted","Data":"8b429c6c924308b59d188fd8ff526788e7cd56cbcfaefd0fa86290211e247eda"}
Jul 20 22:23:39.180357 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:39.180320    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-jtsls" podStartSLOduration=1.5943639840000001 podStartE2EDuration="6.180307353s" totalImagesPullingTime="4.585943369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:39.179532518 +0000 UTC m=+54.957236585" watchObservedRunningTime="2026-07-20 22:23:39.180307353 +0000 UTC m=+54.958011402"
Jul 20 22:23:39.220554 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:39.220341    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7zqcq" podStartSLOduration=1.720788537 podStartE2EDuration="6.220324649s" totalImagesPullingTime="4.499536112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:39.220055157 +0000 UTC m=+54.997759212" watchObservedRunningTime="2026-07-20 22:23:39.220324649 +0000 UTC m=+54.998028681"
Jul 20 22:23:39.220858 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:39.220657    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h" podStartSLOduration=1.664195135 podStartE2EDuration="6.220648295s" totalImagesPullingTime="4.55645316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:39.198768612 +0000 UTC m=+54.976472665" watchObservedRunningTime="2026-07-20 22:23:39.220648295 +0000 UTC m=+54.998352348"
Jul 20 22:23:39.237297 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:39.237259    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv" podStartSLOduration=1.755402957 podStartE2EDuration="6.237247403s" totalImagesPullingTime="4.481844446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:39.235756333 +0000 UTC m=+55.013460385" watchObservedRunningTime="2026-07-20 22:23:39.237247403 +0000 UTC m=+55.014951455"
Jul 20 22:23:39.341018 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:39.340988    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:39.409479 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:39.409449    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:40.055253 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.055218    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:40.124928 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.124891    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dx845"]
Jul 20 22:23:40.156169 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.156143    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dx845"]
Jul 20 22:23:40.156313 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.156237    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.159886 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.159855    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 20 22:23:40.160021 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.159997    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 20 22:23:40.160516 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.160498    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 20 22:23:40.160615 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.160558    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 20 22:23:40.160694 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.160679    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ftm7s\""
Jul 20 22:23:40.184572 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.184550    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:40.270266 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.270237    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:40.353428 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.353352    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-data-volume\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.353817 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.353776    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.353936 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.353909    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhgjp\" (UniqueName: \"kubernetes.io/projected/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-api-access-jhgjp\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.354006 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.353947    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-crio-socket\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.354060 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.354012    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.454869 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.454826    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhgjp\" (UniqueName: \"kubernetes.io/projected/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-api-access-jhgjp\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.454869 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.454864    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-crio-socket\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.455073 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.454903    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.455073 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.454950    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-data-volume\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.455073 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.455033    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-crio-socket\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.455073 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:40.455042    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 22:23:40.455073 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.455070    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.455340 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:40.455112    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm podName:fa7d4497-7ffe-4428-bb59-a6a68a97cfd6 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:40.955097263 +0000 UTC m=+56.732801293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dx845" (UID: "fa7d4497-7ffe-4428-bb59-a6a68a97cfd6") : configmap "kube-rbac-proxy" not found
Jul 20 22:23:40.455340 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:40.455201    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 22:23:40.455340 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:40.455260    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls podName:fa7d4497-7ffe-4428-bb59-a6a68a97cfd6 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:40.955246826 +0000 UTC m=+56.732950856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dx845" (UID: "fa7d4497-7ffe-4428-bb59-a6a68a97cfd6") : secret "insights-runtime-extractor-tls" not found
Jul 20 22:23:40.455476 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.455382    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-data-volume\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.463434 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.463404    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhgjp\" (UniqueName: \"kubernetes.io/projected/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-api-access-jhgjp\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.567951 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.567922    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:40.669391 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.669311    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:40.959101 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.959070    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.959270 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:40.959136    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:40.959270 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:40.959215    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 22:23:40.959270 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:40.959222    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 22:23:40.959270 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:40.959265    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm podName:fa7d4497-7ffe-4428-bb59-a6a68a97cfd6 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:41.959251708 +0000 UTC m=+57.736955738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dx845" (UID: "fa7d4497-7ffe-4428-bb59-a6a68a97cfd6") : configmap "kube-rbac-proxy" not found
Jul 20 22:23:40.959404 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:40.959284    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls podName:fa7d4497-7ffe-4428-bb59-a6a68a97cfd6 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:41.959271476 +0000 UTC m=+57.736975510 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dx845" (UID: "fa7d4497-7ffe-4428-bb59-a6a68a97cfd6") : secret "insights-runtime-extractor-tls" not found
Jul 20 22:23:41.042944 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:41.042909    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:41.214985 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:41.214902    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:41.481142 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:41.481070    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:41.564791 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:41.564754    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:41.564958 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:41.564801    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:41.564958 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:41.564837    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:41.564958 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:41.564899    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:41.564958 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.564903    2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 20 22:23:41.564958 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:41.564938    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6mrqc\" (UID: \"65f38e33-fccd-4881-a0e1-466815c69780\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"
Jul 20 22:23:41.564958 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.564957    2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 20 22:23:41.565251 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.564973    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs podName:2e863cc6-8c15-4725-a3b5-9748719edb3c nodeName:}" failed. No retries permitted until 2026-07-20 22:23:49.564952582 +0000 UTC m=+65.342656631 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs") pod "router-default-54dbf84666-xw84d" (UID: "2e863cc6-8c15-4725-a3b5-9748719edb3c") : secret "router-metrics-certs-default" not found
Jul 20 22:23:41.565251 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.564993    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle podName:2e863cc6-8c15-4725-a3b5-9748719edb3c nodeName:}" failed. No retries permitted until 2026-07-20 22:23:49.564984612 +0000 UTC m=+65.342688656 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle") pod "router-default-54dbf84666-xw84d" (UID: "2e863cc6-8c15-4725-a3b5-9748719edb3c") : configmap references non-existent config key: service-ca.crt
Jul 20 22:23:41.565251 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.565011    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls podName:e9e5fca5-4619-48ab-b7a7-51ac2743a899 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:49.565002532 +0000 UTC m=+65.342706562 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9mdsd" (UID: "e9e5fca5-4619-48ab-b7a7-51ac2743a899") : secret "cluster-monitoring-operator-tls" not found
Jul 20 22:23:41.565251 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.565039    2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 20 22:23:41.565251 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.565106    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls podName:65f38e33-fccd-4881-a0e1-466815c69780 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:49.565089159 +0000 UTC m=+65.342793193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6mrqc" (UID: "65f38e33-fccd-4881-a0e1-466815c69780") : secret "samples-operator-tls" not found
Jul 20 22:23:41.565251 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.565042    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 22:23:41.565251 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.565130    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f4fc9474c-6wjcg: secret "image-registry-tls" not found
Jul 20 22:23:41.565251 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.565167    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls podName:4210e665-12b0-46da-926b-0fa1822f9b63 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:49.565157123 +0000 UTC m=+65.342861159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls") pod "image-registry-6f4fc9474c-6wjcg" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63") : secret "image-registry-tls" not found
Jul 20 22:23:41.968131 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:41.968091    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:41.968319 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:41.968194    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:41.968319 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.968238    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 22:23:41.968319 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.968319    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls podName:fa7d4497-7ffe-4428-bb59-a6a68a97cfd6 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:43.968302625 +0000 UTC m=+59.746006655 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dx845" (UID: "fa7d4497-7ffe-4428-bb59-a6a68a97cfd6") : secret "insights-runtime-extractor-tls" not found
Jul 20 22:23:41.968465 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.968319    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 22:23:41.968465 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:41.968384    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm podName:fa7d4497-7ffe-4428-bb59-a6a68a97cfd6 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:43.968364304 +0000 UTC m=+59.746068356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dx845" (UID: "fa7d4497-7ffe-4428-bb59-a6a68a97cfd6") : configmap "kube-rbac-proxy" not found
Jul 20 22:23:42.310830 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:42.310750    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bw9fm_b94e391c-6611-4879-a037-77852e20a2af/dns-node-resolver/0.log"
Jul 20 22:23:42.689181 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:42.689105    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:42.908883 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:42.908848    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:42.979168 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:42.979140    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:43.511784 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:43.511755    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7vxbl_07ea4c4b-623c-44c9-a630-22f2cf3f91f0/node-ca/0.log"
Jul 20 22:23:43.899166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:43.899091    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:43.985547 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:43.985516    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:43.985683 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:43.985577    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:43.985683 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:43.985669    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 22:23:43.985792 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:43.985684    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 22:23:43.985792 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:43.985742    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls podName:fa7d4497-7ffe-4428-bb59-a6a68a97cfd6 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:47.98572788 +0000 UTC m=+63.763431923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dx845" (UID: "fa7d4497-7ffe-4428-bb59-a6a68a97cfd6") : secret "insights-runtime-extractor-tls" not found
Jul 20 22:23:43.985792 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:43.985758    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm podName:fa7d4497-7ffe-4428-bb59-a6a68a97cfd6 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:47.985751383 +0000 UTC m=+63.763455413 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dx845" (UID: "fa7d4497-7ffe-4428-bb59-a6a68a97cfd6") : configmap "kube-rbac-proxy" not found
Jul 20 22:23:44.109267 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:44.109242    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-548zv_c5302090-68d3-401b-b54f-dd5c292e7d10/serve-healthcheck-canary/0.log"
Jul 20 22:23:44.912929 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:44.912894    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-87h8h_22aad18a-8d9e-4509-9773-5906515b3782/kube-storage-version-migrator-operator/0.log"
Jul 20 22:23:46.085978 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:46.085947    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h8985"
Jul 20 22:23:46.732072 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:46.732035    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:46.796899 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:46.796869    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:47.062750 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:47.062663    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:47.308582 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:47.308552    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:47.401632 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:47.401551    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:47.502085 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:47.502046    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:48.019377 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:48.019330    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:48.019547 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:48.019455    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 22:23:48.019547 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:48.019485    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:48.019547 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:48.019529    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm podName:fa7d4497-7ffe-4428-bb59-a6a68a97cfd6 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:56.01951241 +0000 UTC m=+71.797216459 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dx845" (UID: "fa7d4497-7ffe-4428-bb59-a6a68a97cfd6") : configmap "kube-rbac-proxy" not found
Jul 20 22:23:48.019655 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:48.019584    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 22:23:48.019655 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:48.019636    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls podName:fa7d4497-7ffe-4428-bb59-a6a68a97cfd6 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:56.019623215 +0000 UTC m=+71.797327265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dx845" (UID: "fa7d4497-7ffe-4428-bb59-a6a68a97cfd6") : secret "insights-runtime-extractor-tls" not found
Jul 20 22:23:48.053621 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:48.053595    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:48.851244 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:48.851210    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:49.244794 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:49.244764    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:49.453336 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:49.453303    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:49.633345 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:49.633266    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:49.633345 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:49.633324    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:23:49.633509 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:49.633351    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6mrqc\" (UID: \"65f38e33-fccd-4881-a0e1-466815c69780\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"
Jul 20 22:23:49.633509 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:49.633399    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:23:49.633509 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:49.633441    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle podName:2e863cc6-8c15-4725-a3b5-9748719edb3c nodeName:}" failed. No retries permitted until 2026-07-20 22:24:05.633422483 +0000 UTC m=+81.411126516 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle") pod "router-default-54dbf84666-xw84d" (UID: "2e863cc6-8c15-4725-a3b5-9748719edb3c") : configmap references non-existent config key: service-ca.crt
Jul 20 22:23:49.633509 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:49.633491    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 22:23:49.633509 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:49.633509    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f4fc9474c-6wjcg: secret "image-registry-tls" not found
Jul 20 22:23:49.633662 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:49.633546    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls podName:4210e665-12b0-46da-926b-0fa1822f9b63 nodeName:}" failed. No retries permitted until 2026-07-20 22:24:05.633534402 +0000 UTC m=+81.411238432 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls") pod "image-registry-6f4fc9474c-6wjcg" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63") : secret "image-registry-tls" not found
Jul 20 22:23:49.633711 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:49.633691    2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 20 22:23:49.633782 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:49.633731    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:23:49.633817 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:49.633784    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs podName:2e863cc6-8c15-4725-a3b5-9748719edb3c nodeName:}" failed. No retries permitted until 2026-07-20 22:24:05.633769463 +0000 UTC m=+81.411473503 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs") pod "router-default-54dbf84666-xw84d" (UID: "2e863cc6-8c15-4725-a3b5-9748719edb3c") : secret "router-metrics-certs-default" not found
Jul 20 22:23:49.633860 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:49.633818    2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 20 22:23:49.633895 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:49.633889    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls podName:e9e5fca5-4619-48ab-b7a7-51ac2743a899 nodeName:}" failed. No retries permitted until 2026-07-20 22:24:05.633876022 +0000 UTC m=+81.411580057 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9mdsd" (UID: "e9e5fca5-4619-48ab-b7a7-51ac2743a899") : secret "cluster-monitoring-operator-tls" not found
Jul 20 22:23:49.635868 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:49.635851    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65f38e33-fccd-4881-a0e1-466815c69780-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6mrqc\" (UID: \"65f38e33-fccd-4881-a0e1-466815c69780\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"
Jul 20 22:23:49.644169 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:49.644146    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:49.926010 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:49.925924    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f82s7\""
Jul 20 22:23:49.934086 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:49.934061    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"
Jul 20 22:23:50.052113 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.052083    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:50.056885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.056826    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc"]
Jul 20 22:23:50.204119 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.203279    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc" event={"ID":"65f38e33-fccd-4881-a0e1-466815c69780","Type":"ContainerStarted","Data":"6ece575d6b0d9609c8dd1bc4f4ff4c6f76f1b546cae7490112910cc0125271ba"}
Jul 20 22:23:50.205524 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.205493    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:50.364710 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.364678    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:50.465510 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.465419    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:50.526640 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.526602    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:50.539722 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.539698    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fprgn\" (UniqueName: \"kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn\") pod \"network-check-target-tzv8r\" (UID: \"63b300d3-efba-49ec-8760-7b84c44010d2\") " pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:50.542241 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.542220    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fprgn\" (UniqueName: \"kubernetes.io/projected/63b300d3-efba-49ec-8760-7b84c44010d2-kube-api-access-fprgn\") pod \"network-check-target-tzv8r\" (UID: \"63b300d3-efba-49ec-8760-7b84c44010d2\") " pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:50.640594 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.640565    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:23:50.640728 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.640681    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:50.642994 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.642978    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 22:23:50.643052 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.643035    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 22:23:50.651148 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:50.651134    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 20 22:23:50.651196 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:50.651182    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs podName:116caf25-c9e5-47da-a793-bf52c4aa1f5e nodeName:}" failed. No retries permitted until 2026-07-20 22:24:54.651168129 +0000 UTC m=+130.428872158 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs") pod "network-metrics-daemon-m5bvh" (UID: "116caf25-c9e5-47da-a793-bf52c4aa1f5e") : secret "metrics-daemon-secret" not found
Jul 20 22:23:50.653105 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.653081    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7df0b06-2bc3-45b8-8443-6c4583b2ca28-original-pull-secret\") pod \"global-pull-secret-syncer-89dp7\" (UID: \"d7df0b06-2bc3-45b8-8443-6c4583b2ca28\") " pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:50.658069 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.658054    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-89dp7"
Jul 20 22:23:50.666130 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.666112    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vv84p\""
Jul 20 22:23:50.674933 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.674916    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:50.792788 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.792551    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-89dp7"]
Jul 20 22:23:50.795031 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:23:50.794988    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7df0b06_2bc3_45b8_8443_6c4583b2ca28.slice/crio-c21ed008bd44b455bcfe9345c02415a4c3db373bb0ae08ae7acc6ceaa6361ffb WatchSource:0}: Error finding container c21ed008bd44b455bcfe9345c02415a4c3db373bb0ae08ae7acc6ceaa6361ffb: Status 404 returned error can't find the container with id c21ed008bd44b455bcfe9345c02415a4c3db373bb0ae08ae7acc6ceaa6361ffb
Jul 20 22:23:50.811797 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.811730    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tzv8r"]
Jul 20 22:23:50.811966 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.811939    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:50.814838 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:23:50.814814    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b300d3_efba_49ec_8760_7b84c44010d2.slice/crio-2cfbf00eb697ab3ae8f46cc7acf6480bfe53356f3b38980288f74d524767181c WatchSource:0}: Error finding container 2cfbf00eb697ab3ae8f46cc7acf6480bfe53356f3b38980288f74d524767181c: Status 404 returned error can't find the container with id 2cfbf00eb697ab3ae8f46cc7acf6480bfe53356f3b38980288f74d524767181c
Jul 20 22:23:50.908331 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:50.908306    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:51.051267 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:51.051177    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:51.113054 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:51.113020    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:51.208206 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:51.208151    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tzv8r" event={"ID":"63b300d3-efba-49ec-8760-7b84c44010d2","Type":"ContainerStarted","Data":"c6e4e6ce408ef40f63531af0b9ea6d3b87fee14a23889c71cf415c70f63b56fa"}
Jul 20 22:23:51.208206 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:51.208206    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tzv8r" event={"ID":"63b300d3-efba-49ec-8760-7b84c44010d2","Type":"ContainerStarted","Data":"2cfbf00eb697ab3ae8f46cc7acf6480bfe53356f3b38980288f74d524767181c"}
Jul 20 22:23:51.208552 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:51.208251    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:23:51.209335 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:51.209305    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-89dp7" event={"ID":"d7df0b06-2bc3-45b8-8443-6c4583b2ca28","Type":"ContainerStarted","Data":"c21ed008bd44b455bcfe9345c02415a4c3db373bb0ae08ae7acc6ceaa6361ffb"}
Jul 20 22:23:51.227359 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:51.227283    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tzv8r" podStartSLOduration=67.227263336 podStartE2EDuration="1m7.227263336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:22:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:51.226013395 +0000 UTC m=+67.003717448" watchObservedRunningTime="2026-07-20 22:23:51.227263336 +0000 UTC m=+67.004967391"
Jul 20 22:23:51.641269 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:51.641233    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:51.865358 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:51.865315    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:52.215647 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:52.215613    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc" event={"ID":"65f38e33-fccd-4881-a0e1-466815c69780","Type":"ContainerStarted","Data":"41ab68e562a5db5b3bbd96c6d8a08cbb471f8a4cbd45649db800edf33aaa81e3"}
Jul 20 22:23:52.305278 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:52.305247    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:52.458993 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:52.458958    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:52.557449 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:52.557417    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:52.559777 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:52.559751    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69c9b463-2c26-4b92-b415-c5bf20afb209-metrics-tls\") pod \"dns-default-x6w7q\" (UID: \"69c9b463-2c26-4b92-b415-c5bf20afb209\") " pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:52.785760 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:52.785672    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vb99p\""
Jul 20 22:23:52.793493 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:52.793467    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:52.841077 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:52.841041    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:52.935329 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:52.935306    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x6w7q"]
Jul 20 22:23:52.938263 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:23:52.938229    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69c9b463_2c26_4b92_b415_c5bf20afb209.slice/crio-3f33ee27af296783981cb88f3820968db2a9d23c25887755b0d6151bc082fc16 WatchSource:0}: Error finding container 3f33ee27af296783981cb88f3820968db2a9d23c25887755b0d6151bc082fc16: Status 404 returned error can't find the container with id 3f33ee27af296783981cb88f3820968db2a9d23c25887755b0d6151bc082fc16
Jul 20 22:23:53.148661 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:53.148584    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:53.220643 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:53.220606    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc" event={"ID":"65f38e33-fccd-4881-a0e1-466815c69780","Type":"ContainerStarted","Data":"8ed3a2f2b9292dde1d5e09cb20e5c77e903a8a667b809230bc45e41301e7038d"}
Jul 20 22:23:53.221830 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:53.221801    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x6w7q" event={"ID":"69c9b463-2c26-4b92-b415-c5bf20afb209","Type":"ContainerStarted","Data":"3f33ee27af296783981cb88f3820968db2a9d23c25887755b0d6151bc082fc16"}
Jul 20 22:23:53.249484 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:53.249427    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6mrqc" podStartSLOduration=18.212519044 podStartE2EDuration="20.249409566s" totalImagesPullingTime="2.036890522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:53.248846194 +0000 UTC m=+69.026550247" watchObservedRunningTime="2026-07-20 22:23:53.249409566 +0000 UTC m=+69.027113619"
Jul 20 22:23:53.288782 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:53.288711    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:53.641629 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:53.641588    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:54.043892 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:54.043811    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:54.241960 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:54.241930    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:54.439488 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:54.439449    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:54.644860 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:54.644782    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:55.846464 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:55.846429    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:56.041802 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:56.041752    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:56.090134 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:56.090109    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:56.090243 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:56.090161    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:56.090282 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:56.090248    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 22:23:56.090317 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:23:56.090298    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls podName:fa7d4497-7ffe-4428-bb59-a6a68a97cfd6 nodeName:}" failed. No retries permitted until 2026-07-20 22:24:12.090284194 +0000 UTC m=+87.867988223 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dx845" (UID: "fa7d4497-7ffe-4428-bb59-a6a68a97cfd6") : secret "insights-runtime-extractor-tls" not found
Jul 20 22:23:56.090624 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:56.090607    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:23:56.234952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:56.234917    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-89dp7" event={"ID":"d7df0b06-2bc3-45b8-8443-6c4583b2ca28","Type":"ContainerStarted","Data":"a8dfe3a9c9f4df39c19a5e7e179fc4317e7fe85850f47d0e89b6221814abb8dc"}
Jul 20 22:23:56.236588 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:56.236567    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x6w7q" event={"ID":"69c9b463-2c26-4b92-b415-c5bf20afb209","Type":"ContainerStarted","Data":"8c02856a20b017bc4551c49aa13a35e52df4e8d2c1fdb5122692ac18e6b8dd3b"}
Jul 20 22:23:56.236701 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:56.236603    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x6w7q" event={"ID":"69c9b463-2c26-4b92-b415-c5bf20afb209","Type":"ContainerStarted","Data":"6036c6caff7b16ed4aa2be63448fbdaed4ef8adedc09eea2df99aa2a0695958d"}
Jul 20 22:23:56.236701 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:56.236697    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:23:56.260310 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:56.260263    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-89dp7" podStartSLOduration=66.436404888 podStartE2EDuration="1m11.260250954s" totalImagesPullingTime="4.823846066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:56.259045165 +0000 UTC m=+72.036749216" watchObservedRunningTime="2026-07-20 22:23:56.260250954 +0000 UTC m=+72.037955006"
Jul 20 22:23:56.261194 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:56.261160    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:56.280812 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:56.280764    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-x6w7q" podStartSLOduration=33.596103873 podStartE2EDuration="36.280751456s" totalImagesPullingTime="2.684647583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:56.280041525 +0000 UTC m=+72.057745569" watchObservedRunningTime="2026-07-20 22:23:56.280751456 +0000 UTC m=+72.058455508"
Jul 20 22:23:56.439406 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:56.439361    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:56.646021 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:56.645929    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:57.252391 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:57.252352    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:57.458058 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:57.457990    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:58.124612 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:58.124559    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:23:58.282863 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:23:58.282808    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:04.766914 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.766882    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"]
Jul 20 22:24:04.769250 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.769228    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.771540 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.771518    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 20 22:24:04.771540 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.771530    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 20 22:24:04.771711 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.771607    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 20 22:24:04.771711 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.771627    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 20 22:24:04.772258 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.772239    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 20 22:24:04.772351 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.772254    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 20 22:24:04.772409 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.772394    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 20 22:24:04.782038 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.782015    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"]
Jul 20 22:24:04.849112 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.849085    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f4fc9474c-6wjcg"]
Jul 20 22:24:04.849302 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:24:04.849285    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg" podUID="4210e665-12b0-46da-926b-0fa1822f9b63"
Jul 20 22:24:04.853897 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.853840    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:04.857908 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.857885    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-hub\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.857993 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.857929    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.858052 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.857999    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.858052 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.858026    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.858150 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.858059    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-ca\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.858150 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.858140    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22xwb\" (UniqueName: \"kubernetes.io/projected/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-kube-api-access-22xwb\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.892601 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.892574    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79d9754994-wsl8w"]
Jul 20 22:24:04.894632 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.894617    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:04.911405 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.911378    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79d9754994-wsl8w"]
Jul 20 22:24:04.953458 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.953423    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:04.958485 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958467    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-hub\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.958551 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958499    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.958551 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958528    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ec36065-a4fb-466a-9aa6-df816c8c8669-installation-pull-secrets\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:04.958675 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958567    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.958675 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958594    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.958675 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958617    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ec36065-a4fb-466a-9aa6-df816c8c8669-ca-trust-extracted\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:04.958675 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958640    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm9kr\" (UniqueName: \"kubernetes.io/projected/5ec36065-a4fb-466a-9aa6-df816c8c8669-kube-api-access-wm9kr\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:04.958675 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958669    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-ca\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.958957 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958698    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ec36065-a4fb-466a-9aa6-df816c8c8669-bound-sa-token\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:04.958957 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958761    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ec36065-a4fb-466a-9aa6-df816c8c8669-registry-certificates\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:04.958957 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958796    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ec36065-a4fb-466a-9aa6-df816c8c8669-image-registry-private-configuration\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:04.958957 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958841    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22xwb\" (UniqueName: \"kubernetes.io/projected/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-kube-api-access-22xwb\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.958957 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958876    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ec36065-a4fb-466a-9aa6-df816c8c8669-registry-tls\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:04.958957 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.958919    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ec36065-a4fb-466a-9aa6-df816c8c8669-trusted-ca\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:04.959374 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.959320    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.961272 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.961250    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-ca\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.961545 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.961525    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-hub\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.961595 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.961561    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.961689 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.961674    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:04.969422 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:04.969401    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22xwb\" (UniqueName: \"kubernetes.io/projected/2c9bea51-7bae-45b8-ae8b-3830f9fc78d9-kube-api-access-22xwb\") pod \"cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg\" (UID: \"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:05.059686 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.059623    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ec36065-a4fb-466a-9aa6-df816c8c8669-installation-pull-secrets\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.059686 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.059658    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ec36065-a4fb-466a-9aa6-df816c8c8669-ca-trust-extracted\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.059686 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.059675    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm9kr\" (UniqueName: \"kubernetes.io/projected/5ec36065-a4fb-466a-9aa6-df816c8c8669-kube-api-access-wm9kr\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.059927 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.059710    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ec36065-a4fb-466a-9aa6-df816c8c8669-bound-sa-token\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.059927 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.059770    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ec36065-a4fb-466a-9aa6-df816c8c8669-registry-certificates\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.059927 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.059800    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ec36065-a4fb-466a-9aa6-df816c8c8669-image-registry-private-configuration\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.060096 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.060024    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ec36065-a4fb-466a-9aa6-df816c8c8669-registry-tls\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.060199 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.060090    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ec36065-a4fb-466a-9aa6-df816c8c8669-ca-trust-extracted\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.060199 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.060146    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ec36065-a4fb-466a-9aa6-df816c8c8669-trusted-ca\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.060755 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.060707    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ec36065-a4fb-466a-9aa6-df816c8c8669-registry-certificates\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.061041 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.061022    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ec36065-a4fb-466a-9aa6-df816c8c8669-trusted-ca\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.062166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.062148    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ec36065-a4fb-466a-9aa6-df816c8c8669-installation-pull-secrets\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.062494 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.062474    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ec36065-a4fb-466a-9aa6-df816c8c8669-image-registry-private-configuration\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.062624 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.062602    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ec36065-a4fb-466a-9aa6-df816c8c8669-registry-tls\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.063314 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.063295    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:05.072804 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.072779    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ec36065-a4fb-466a-9aa6-df816c8c8669-bound-sa-token\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.073159 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.073143    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm9kr\" (UniqueName: \"kubernetes.io/projected/5ec36065-a4fb-466a-9aa6-df816c8c8669-kube-api-access-wm9kr\") pod \"image-registry-79d9754994-wsl8w\" (UID: \"5ec36065-a4fb-466a-9aa6-df816c8c8669\") " pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.093168 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.093149    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"
Jul 20 22:24:05.167101 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.166943    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:05.208136 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.208105    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-42787\""
Jul 20 22:24:05.214334 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.214310    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:05.236270 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.236248    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg"]
Jul 20 22:24:05.238286 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:24:05.238253    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c9bea51_7bae_45b8_ae8b_3830f9fc78d9.slice/crio-fe4e9c8cd85fdbc3dcbe5ec1247fc52e80b5cee047fcb80f2ec587da73e4c755 WatchSource:0}: Error finding container fe4e9c8cd85fdbc3dcbe5ec1247fc52e80b5cee047fcb80f2ec587da73e4c755: Status 404 returned error can't find the container with id fe4e9c8cd85fdbc3dcbe5ec1247fc52e80b5cee047fcb80f2ec587da73e4c755
Jul 20 22:24:05.266439 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.266415    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:24:05.266542 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.266422    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg" event={"ID":"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9","Type":"ContainerStarted","Data":"fe4e9c8cd85fdbc3dcbe5ec1247fc52e80b5cee047fcb80f2ec587da73e4c755"}
Jul 20 22:24:05.278847 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.278825    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:24:05.337157 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.337082    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:05.357289 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.357266    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79d9754994-wsl8w"]
Jul 20 22:24:05.359384 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:24:05.359360    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ec36065_a4fb_466a_9aa6_df816c8c8669.slice/crio-c2d6ff316d528528fbc377c2835c6cea3a395b3a46ae9b4c6bc4e75a42f03694 WatchSource:0}: Error finding container c2d6ff316d528528fbc377c2835c6cea3a395b3a46ae9b4c6bc4e75a42f03694: Status 404 returned error can't find the container with id c2d6ff316d528528fbc377c2835c6cea3a395b3a46ae9b4c6bc4e75a42f03694
Jul 20 22:24:05.362838 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.362680    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6skj\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-kube-api-access-r6skj\") pod \"4210e665-12b0-46da-926b-0fa1822f9b63\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") "
Jul 20 22:24:05.362838 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.362730    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4210e665-12b0-46da-926b-0fa1822f9b63-installation-pull-secrets\") pod \"4210e665-12b0-46da-926b-0fa1822f9b63\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") "
Jul 20 22:24:05.362838 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.362767    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-bound-sa-token\") pod \"4210e665-12b0-46da-926b-0fa1822f9b63\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") "
Jul 20 22:24:05.362838 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.362791    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4210e665-12b0-46da-926b-0fa1822f9b63-ca-trust-extracted\") pod \"4210e665-12b0-46da-926b-0fa1822f9b63\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") "
Jul 20 22:24:05.362838 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.362806    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4210e665-12b0-46da-926b-0fa1822f9b63-trusted-ca\") pod \"4210e665-12b0-46da-926b-0fa1822f9b63\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") "
Jul 20 22:24:05.363101 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.362872    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4210e665-12b0-46da-926b-0fa1822f9b63-image-registry-private-configuration\") pod \"4210e665-12b0-46da-926b-0fa1822f9b63\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") "
Jul 20 22:24:05.363101 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.362924    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4210e665-12b0-46da-926b-0fa1822f9b63-registry-certificates\") pod \"4210e665-12b0-46da-926b-0fa1822f9b63\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") "
Jul 20 22:24:05.363101 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.363088    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4210e665-12b0-46da-926b-0fa1822f9b63-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4210e665-12b0-46da-926b-0fa1822f9b63" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 22:24:05.363255 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.363209    2550 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4210e665-12b0-46da-926b-0fa1822f9b63-ca-trust-extracted\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:24:05.363396 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.363361    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4210e665-12b0-46da-926b-0fa1822f9b63-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4210e665-12b0-46da-926b-0fa1822f9b63" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 22:24:05.363471 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.363451    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4210e665-12b0-46da-926b-0fa1822f9b63-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4210e665-12b0-46da-926b-0fa1822f9b63" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 22:24:05.365529 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.365501    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4210e665-12b0-46da-926b-0fa1822f9b63-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4210e665-12b0-46da-926b-0fa1822f9b63" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 22:24:05.365621 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.365503    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-kube-api-access-r6skj" (OuterVolumeSpecName: "kube-api-access-r6skj") pod "4210e665-12b0-46da-926b-0fa1822f9b63" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63"). InnerVolumeSpecName "kube-api-access-r6skj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 22:24:05.365621 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.365539    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4210e665-12b0-46da-926b-0fa1822f9b63" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 22:24:05.365621 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.365558    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4210e665-12b0-46da-926b-0fa1822f9b63-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4210e665-12b0-46da-926b-0fa1822f9b63" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 22:24:05.464607 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.464575    2550 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4210e665-12b0-46da-926b-0fa1822f9b63-registry-certificates\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:24:05.464607 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.464601    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6skj\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-kube-api-access-r6skj\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:24:05.464607 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.464611    2550 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4210e665-12b0-46da-926b-0fa1822f9b63-installation-pull-secrets\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:24:05.464807 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.464619    2550 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-bound-sa-token\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:24:05.464807 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.464628    2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4210e665-12b0-46da-926b-0fa1822f9b63-trusted-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:24:05.464807 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.464636    2550 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4210e665-12b0-46da-926b-0fa1822f9b63-image-registry-private-configuration\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:24:05.667233 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.667141    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:24:05.667382 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.667250    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:24:05.667382 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.667289    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:24:05.667382 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.667331    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:24:05.668092 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.668055    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e863cc6-8c15-4725-a3b5-9748719edb3c-service-ca-bundle\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:24:05.670331 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.670309    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls\") pod \"image-registry-6f4fc9474c-6wjcg\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") " pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:24:05.670432 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.670327    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e5fca5-4619-48ab-b7a7-51ac2743a899-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mdsd\" (UID: \"e9e5fca5-4619-48ab-b7a7-51ac2743a899\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:24:05.670432 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.670327    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e863cc6-8c15-4725-a3b5-9748719edb3c-metrics-certs\") pod \"router-default-54dbf84666-xw84d\" (UID: \"2e863cc6-8c15-4725-a3b5-9748719edb3c\") " pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:24:05.768637 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.768596    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls\") pod \"4210e665-12b0-46da-926b-0fa1822f9b63\" (UID: \"4210e665-12b0-46da-926b-0fa1822f9b63\") "
Jul 20 22:24:05.771205 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.771161    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4210e665-12b0-46da-926b-0fa1822f9b63" (UID: "4210e665-12b0-46da-926b-0fa1822f9b63"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 22:24:05.870229 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.870182    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4210e665-12b0-46da-926b-0fa1822f9b63-registry-tls\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:24:05.875353 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.875320    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-t9kv4\""
Jul 20 22:24:05.883303 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.883278    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"
Jul 20 22:24:05.901932 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.901913    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-lfzmn\""
Jul 20 22:24:05.910849 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:05.910828    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:24:06.069985 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.069906    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd"]
Jul 20 22:24:06.075592 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:24:06.075547    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e5fca5_4619_48ab_b7a7_51ac2743a899.slice/crio-30986764f39f35cbe2c1cdf5a6ec2233c2a077fc15bc4ed5003715e2e514dfea WatchSource:0}: Error finding container 30986764f39f35cbe2c1cdf5a6ec2233c2a077fc15bc4ed5003715e2e514dfea: Status 404 returned error can't find the container with id 30986764f39f35cbe2c1cdf5a6ec2233c2a077fc15bc4ed5003715e2e514dfea
Jul 20 22:24:06.094820 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.094784    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54dbf84666-xw84d"]
Jul 20 22:24:06.098581 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:24:06.098548    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e863cc6_8c15_4725_a3b5_9748719edb3c.slice/crio-10e7b37b263acdba1f7de400787daaee898603effa5ed80a232794f23a5b87af WatchSource:0}: Error finding container 10e7b37b263acdba1f7de400787daaee898603effa5ed80a232794f23a5b87af: Status 404 returned error can't find the container with id 10e7b37b263acdba1f7de400787daaee898603effa5ed80a232794f23a5b87af
Jul 20 22:24:06.241791 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.241763    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x6w7q"
Jul 20 22:24:06.289745 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.278296    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd" event={"ID":"e9e5fca5-4619-48ab-b7a7-51ac2743a899","Type":"ContainerStarted","Data":"30986764f39f35cbe2c1cdf5a6ec2233c2a077fc15bc4ed5003715e2e514dfea"}
Jul 20 22:24:06.289745 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.282880    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79d9754994-wsl8w" event={"ID":"5ec36065-a4fb-466a-9aa6-df816c8c8669","Type":"ContainerStarted","Data":"d6681ff0f728f5176529aae3df902f5f904a01316d1a67e95baa464cdba0626d"}
Jul 20 22:24:06.289745 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.282929    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79d9754994-wsl8w" event={"ID":"5ec36065-a4fb-466a-9aa6-df816c8c8669","Type":"ContainerStarted","Data":"c2d6ff316d528528fbc377c2835c6cea3a395b3a46ae9b4c6bc4e75a42f03694"}
Jul 20 22:24:06.289745 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.283460    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:06.289745 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.285143    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f4fc9474c-6wjcg"
Jul 20 22:24:06.289745 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.289135    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54dbf84666-xw84d" event={"ID":"2e863cc6-8c15-4725-a3b5-9748719edb3c","Type":"ContainerStarted","Data":"25c6855ff7af1e5f99c3e1dc06aec5d7ebd5f3e1d66023028e6dedb53c06b876"}
Jul 20 22:24:06.289745 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.289168    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54dbf84666-xw84d" event={"ID":"2e863cc6-8c15-4725-a3b5-9748719edb3c","Type":"ContainerStarted","Data":"10e7b37b263acdba1f7de400787daaee898603effa5ed80a232794f23a5b87af"}
Jul 20 22:24:06.418466 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.418434    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f4fc9474c-6wjcg"]
Jul 20 22:24:06.453950 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.453884    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6f4fc9474c-6wjcg"]
Jul 20 22:24:06.569269 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.569176    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-54dbf84666-xw84d" podStartSLOduration=33.569157208 podStartE2EDuration="33.569157208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:06.568116488 +0000 UTC m=+82.345820534" watchObservedRunningTime="2026-07-20 22:24:06.569157208 +0000 UTC m=+82.346861261"
Jul 20 22:24:06.569582 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.569338    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-79d9754994-wsl8w" podStartSLOduration=2.56933078 podStartE2EDuration="2.56933078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:24:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:06.510825119 +0000 UTC m=+82.288529172" watchObservedRunningTime="2026-07-20 22:24:06.56933078 +0000 UTC m=+82.347034832"
Jul 20 22:24:06.649801 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.649660    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:06.741594 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.741526    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:06.846202 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.846043    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4210e665-12b0-46da-926b-0fa1822f9b63" path="/var/lib/kubelet/pods/4210e665-12b0-46da-926b-0fa1822f9b63/volumes"
Jul 20 22:24:06.911967 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.911894    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:24:06.914750 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:06.914672    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:24:07.290345 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:07.290287    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:24:07.291768 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:07.291708    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-54dbf84666-xw84d"
Jul 20 22:24:07.693566 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:07.693492    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:07.758997 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:07.758965    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:07.828393 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:07.828363    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:09.835629 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:09.835599    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:10.004755 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:10.004660    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:10.302063 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:10.301981    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg" event={"ID":"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9","Type":"ContainerStarted","Data":"26dd93e0de7a77dfa26ace2ec30051537d43a41a518805b9f173fd01721a02bc"}
Jul 20 22:24:10.302063 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:10.302025    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg" event={"ID":"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9","Type":"ContainerStarted","Data":"64f2c097a1581940bd552a96d5f2c279b99129464560907c44b4ca72fdffb609"}
Jul 20 22:24:10.302063 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:10.302041    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg" event={"ID":"2c9bea51-7bae-45b8-ae8b-3830f9fc78d9","Type":"ContainerStarted","Data":"d2edac7703d3bed48a47ef904e49cd421394ee97afe76428e4924beb44af5321"}
Jul 20 22:24:10.303352 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:10.303319    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd" event={"ID":"e9e5fca5-4619-48ab-b7a7-51ac2743a899","Type":"ContainerStarted","Data":"d6c2e6e52591246ea4169fc5fd7a74ba2fe5bcdd5c23ae1580a85cb06cd4ecc4"}
Jul 20 22:24:10.323363 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:10.323323    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbfdfc67b-k8dwg" podStartSLOduration=2.105578095 podStartE2EDuration="6.323311451s" totalImagesPullingTime="4.217733356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:24:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:10.322308847 +0000 UTC m=+86.100012899" watchObservedRunningTime="2026-07-20 22:24:10.323311451 +0000 UTC m=+86.101015481"
Jul 20 22:24:10.350088 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:10.350051    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mdsd" podStartSLOduration=33.983847734 podStartE2EDuration="37.350038805s" totalImagesPullingTime="3.366191071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:10.349008366 +0000 UTC m=+86.126712420" watchObservedRunningTime="2026-07-20 22:24:10.350038805 +0000 UTC m=+86.127742857"
Jul 20 22:24:10.621408 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:10.621322    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:11.838806 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:11.838768    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:11.908633 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:11.908601    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:12.129796 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:12.129697    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:24:12.131966 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:12.131946    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fa7d4497-7ffe-4428-bb59-a6a68a97cfd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dx845\" (UID: \"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6\") " pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:24:12.267453 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:12.267421    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ftm7s\""
Jul 20 22:24:12.275813 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:12.275789    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dx845"
Jul 20 22:24:12.403596 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:12.403569    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dx845"]
Jul 20 22:24:12.405875 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:24:12.405851    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa7d4497_7ffe_4428_bb59_a6a68a97cfd6.slice/crio-6a6e5177459e592adab39105765f540bf58cb11fe0a29e4ed7b9934d6567ece5 WatchSource:0}: Error finding container 6a6e5177459e592adab39105765f540bf58cb11fe0a29e4ed7b9934d6567ece5: Status 404 returned error can't find the container with id 6a6e5177459e592adab39105765f540bf58cb11fe0a29e4ed7b9934d6567ece5
Jul 20 22:24:13.316047 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:13.316005    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dx845" event={"ID":"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6","Type":"ContainerStarted","Data":"6a6e5177459e592adab39105765f540bf58cb11fe0a29e4ed7b9934d6567ece5"}
Jul 20 22:24:15.458838 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:15.458759    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:15.559350 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:15.559309    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:15.624664 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:15.624628    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:16.041947 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:16.041906    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:16.122096 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:16.122064    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:16.229920 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:16.229880    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:17.338210 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:17.338178    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dx845" event={"ID":"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6","Type":"ContainerStarted","Data":"2c44576118f0a9ceab2977c5c08f3319b5221df6b50f6d51585c040498a3bf93"}
Jul 20 22:24:19.842425 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:19.842252    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:19.952992 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:19.952949    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:21.153802 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:21.153702    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:21.356546 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:21.356505    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dx845" event={"ID":"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6","Type":"ContainerStarted","Data":"bef22f8309acff65b10d04f17b03bef883f798fe79fa78b9773852c625ce7248"}
Jul 20 22:24:22.219263 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:22.219228    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tzv8r"
Jul 20 22:24:22.826800 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:22.826757    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:22.931640 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:22.931576    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:22.989312 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:22.989251    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:23.182044 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.181945    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:23.468373 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.468338    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:23.537937 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.537905    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gpkdp"]
Jul 20 22:24:23.555746 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.555702    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.561527 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.561481    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 20 22:24:23.562653 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.562476    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k4svg\""
Jul 20 22:24:23.563063 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.563027    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 20 22:24:23.563175 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.563137    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 20 22:24:23.563240 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.563231    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 20 22:24:23.598700 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.598671    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:23.701582 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.701532    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:23.729111 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.729036    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl642\" (UniqueName: \"kubernetes.io/projected/421fb845-fa15-4321-8774-7661b458d95d-kube-api-access-fl642\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.729253 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.729113    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-wtmp\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.729253 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.729141    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-textfile\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.729253 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.729164    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/421fb845-fa15-4321-8774-7661b458d95d-sys\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.729253 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.729189    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/421fb845-fa15-4321-8774-7661b458d95d-metrics-client-ca\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.729253 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.729231    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-tls\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.729454 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.729290    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/421fb845-fa15-4321-8774-7661b458d95d-root\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.729454 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.729312    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-accelerators-collector-config\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.729454 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.729354    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.799254 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.799220    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:23.830326 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.830295    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-tls\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.830495 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.830383    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/421fb845-fa15-4321-8774-7661b458d95d-root\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.830495 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.830415    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-accelerators-collector-config\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.830495 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.830454    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.830610 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.830490    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl642\" (UniqueName: \"kubernetes.io/projected/421fb845-fa15-4321-8774-7661b458d95d-kube-api-access-fl642\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.830610 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.830521    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/421fb845-fa15-4321-8774-7661b458d95d-root\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.830610 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.830552    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-wtmp\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.830610 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.830581    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-textfile\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.830610 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.830608    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/421fb845-fa15-4321-8774-7661b458d95d-sys\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.830882 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.830633    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/421fb845-fa15-4321-8774-7661b458d95d-metrics-client-ca\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.830882 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.830749    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-wtmp\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.831337 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.831273    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/421fb845-fa15-4321-8774-7661b458d95d-metrics-client-ca\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.831337 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.831289    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-textfile\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.831337 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.831296    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/421fb845-fa15-4321-8774-7661b458d95d-sys\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.831571 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.831535    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-accelerators-collector-config\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.833026 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.833002    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-tls\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.833190 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.833165    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/421fb845-fa15-4321-8774-7661b458d95d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.842021 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.841987    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl642\" (UniqueName: \"kubernetes.io/projected/421fb845-fa15-4321-8774-7661b458d95d-kube-api-access-fl642\") pod \"node-exporter-gpkdp\" (UID: \"421fb845-fa15-4321-8774-7661b458d95d\") " pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.867060 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.867039    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gpkdp"
Jul 20 22:24:23.902586 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:23.902554    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:24.142584 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:24.142497    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:24.225649 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:24:24.225614    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod421fb845_fa15_4321_8774_7661b458d95d.slice/crio-162f9bce9605a87efcb29bbdfc7901d017329b28dae3dcb2916c91f931b14222 WatchSource:0}: Error finding container 162f9bce9605a87efcb29bbdfc7901d017329b28dae3dcb2916c91f931b14222: Status 404 returned error can't find the container with id 162f9bce9605a87efcb29bbdfc7901d017329b28dae3dcb2916c91f931b14222
Jul 20 22:24:24.372923 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:24.372892    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gpkdp" event={"ID":"421fb845-fa15-4321-8774-7661b458d95d","Type":"ContainerStarted","Data":"162f9bce9605a87efcb29bbdfc7901d017329b28dae3dcb2916c91f931b14222"}
Jul 20 22:24:24.375085 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:24.375059    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dx845" event={"ID":"fa7d4497-7ffe-4428-bb59-a6a68a97cfd6","Type":"ContainerStarted","Data":"23fc1de1f310b3b4a4014860c082882d2e5d6d2263dc9a44a3a4156a92fe2b3c"}
Jul 20 22:24:24.396826 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:24.396710    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dx845" podStartSLOduration=32.64652499 podStartE2EDuration="44.396694024s" totalImagesPullingTime="11.750169034s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:23:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:24.39552281 +0000 UTC m=+100.173226864" watchObservedRunningTime="2026-07-20 22:24:24.396694024 +0000 UTC m=+100.174398078"
Jul 20 22:24:24.445968 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:24.445933    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:24.565183 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:24.565154    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:24.649497 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:24.649412    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:24.971320 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:24.971286    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:25.385277 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:25.385246    2550 generic.go:356] "Generic (PLEG): container finished" podID="421fb845-fa15-4321-8774-7661b458d95d" containerID="cf391ce5c86edae851a8264a9027172c88d9f6ab948aa04b5344b800d93c5564" exitCode=0
Jul 20 22:24:25.385422 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:25.385331    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gpkdp" event={"ID":"421fb845-fa15-4321-8774-7661b458d95d","Type":"ContainerDied","Data":"cf391ce5c86edae851a8264a9027172c88d9f6ab948aa04b5344b800d93c5564"}
Jul 20 22:24:26.393729 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:26.393681    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gpkdp" event={"ID":"421fb845-fa15-4321-8774-7661b458d95d","Type":"ContainerStarted","Data":"2c32b55a8a7c8455db748479db036d0067669edf36731c006377ee8ccb93ff16"}
Jul 20 22:24:26.394087 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:26.393758    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gpkdp" event={"ID":"421fb845-fa15-4321-8774-7661b458d95d","Type":"ContainerStarted","Data":"f4a4e64fe62fd3bebd16f86ca6246fa0550e6065f41b0549bdc057e3322d7969"}
Jul 20 22:24:26.417931 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:26.417884    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gpkdp" podStartSLOduration=2.486150296 podStartE2EDuration="3.417869671s" totalImagesPullingTime="868.424975ms" totalInitContainerRuntime="63.2944ms" isStatefulPod=true podCreationTimestamp="2026-07-20 22:24:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:26.415486421 +0000 UTC m=+102.193190472" watchObservedRunningTime="2026-07-20 22:24:26.417869671 +0000 UTC m=+102.195573722"
Jul 20 22:24:26.473517 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:26.473485    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:26.576275 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:26.576240    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:26.651378 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:26.651314    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:26.956852 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:26.956821    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:27.059323 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:27.059293    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:27.487012 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:27.486985    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:28.036040 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:28.036005    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:28.104146 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:28.104105    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:28.217918 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:28.217885    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:28.298299 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:28.298236    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79d9754994-wsl8w"
Jul 20 22:24:28.300369 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:28.300342    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:28.435195 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:28.435158    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:28.536843 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:28.536800    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:28.637582 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:28.637500    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:28.700845 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:28.700816    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:29.625246 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:29.625212    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:29.725368 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:29.725339    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:29.814974 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:29.814941    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:29.926518 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:29.926439    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:31.644487 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:31.644450    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:31.817523 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:31.817487    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:31.957400 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:31.957370    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:32.053328 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:32.053288    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:32.418440 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:32.418369    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:32.567669 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:32.567635    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:32.665267 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:32.665236    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:33.066882 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:33.066844    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:33.169353 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:33.169324    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:33.300997 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:33.300965    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:33.481228 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:33.481200    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:33.592568 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:33.592540    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:34.669304 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:34.669270    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:37.271603 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:37.271566    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:37.342892 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:37.342857    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:37.965319 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:37.965282    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:38.054339 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:38.054311    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:38.876937 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:38.876907    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:38.989302 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:38.989272    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:39.432940 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:39.432910    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:39.536684 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:39.536649    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:43.036349 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:43.036313    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:43.137585 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:43.137548    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:49.471759 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:49.471709    2550 generic.go:356] "Generic (PLEG): container finished" podID="22aad18a-8d9e-4509-9773-5906515b3782" containerID="bf1b0a327bf4a21b9d84d1df92218e87ab9927349ed9c3cce70bf58bd912f0c3" exitCode=0
Jul 20 22:24:49.472152 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:49.471775    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h" event={"ID":"22aad18a-8d9e-4509-9773-5906515b3782","Type":"ContainerDied","Data":"bf1b0a327bf4a21b9d84d1df92218e87ab9927349ed9c3cce70bf58bd912f0c3"}
Jul 20 22:24:49.472152 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:49.472095    2550 scope.go:117] "RemoveContainer" containerID="bf1b0a327bf4a21b9d84d1df92218e87ab9927349ed9c3cce70bf58bd912f0c3"
Jul 20 22:24:50.477004 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:50.476972    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-87h8h" event={"ID":"22aad18a-8d9e-4509-9773-5906515b3782","Type":"ContainerStarted","Data":"c984cd2ec9f827de5bbd94571169f71b6cb06ad51a91476ca389e29273de1832"}
Jul 20 22:24:53.843585 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:53.843547    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:53.920905 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:53.920876    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:54.657052 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:54.657018    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:24:54.659180 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:54.659161    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/116caf25-c9e5-47da-a793-bf52c4aa1f5e-metrics-certs\") pod \"network-metrics-daemon-m5bvh\" (UID: \"116caf25-c9e5-47da-a793-bf52c4aa1f5e\") " pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:24:54.852905 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:54.852878    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wpkcl\""
Jul 20 22:24:54.860898 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:54.860877    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m5bvh"
Jul 20 22:24:55.264968 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:55.264937    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m5bvh"]
Jul 20 22:24:55.268055 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:24:55.268023    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod116caf25_c9e5_47da_a793_bf52c4aa1f5e.slice/crio-5460e4b925140cdb33c34f3ff777cf8603da9e095133d536876205c5fb98ee16 WatchSource:0}: Error finding container 5460e4b925140cdb33c34f3ff777cf8603da9e095133d536876205c5fb98ee16: Status 404 returned error can't find the container with id 5460e4b925140cdb33c34f3ff777cf8603da9e095133d536876205c5fb98ee16
Jul 20 22:24:55.494945 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:55.494910    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m5bvh" event={"ID":"116caf25-c9e5-47da-a793-bf52c4aa1f5e","Type":"ContainerStarted","Data":"5460e4b925140cdb33c34f3ff777cf8603da9e095133d536876205c5fb98ee16"}
Jul 20 22:24:56.500139 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:56.500101    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m5bvh" event={"ID":"116caf25-c9e5-47da-a793-bf52c4aa1f5e","Type":"ContainerStarted","Data":"dbddfa5d3256c15444041eeec5c35649f6f8bb8c20d6b0d059cf15892ff5940d"}
Jul 20 22:24:56.500530 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:56.500162    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m5bvh" event={"ID":"116caf25-c9e5-47da-a793-bf52c4aa1f5e","Type":"ContainerStarted","Data":"afa5a5892140a29ff0140741195a55cc2cdd43cb7b22b4d2a0f6f1a32d869af9"}
Jul 20 22:24:56.517490 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:56.517403    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-m5bvh" podStartSLOduration=131.529037823 podStartE2EDuration="2m12.517388334s" totalImagesPullingTime="988.350511ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:56.516556166 +0000 UTC m=+132.294260241" watchObservedRunningTime="2026-07-20 22:24:56.517388334 +0000 UTC m=+132.295092386"
Jul 20 22:24:56.623475 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:56.623446    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:56.686355 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:56.686325    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:56.834169 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:56.834092    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:57.450680 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:57.450646    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:57.550455 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:57.550422    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:58.032386 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:58.032354    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:24:58.101458 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:24:58.101422    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:03.091530 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:03.091498    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:03.191646 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:03.191616    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:10.550622 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:10.550587    2550 generic.go:356] "Generic (PLEG): container finished" podID="21d248fd-bc94-4517-86e4-79e38827640e" containerID="15674c5073c1e70e25a6bde9dc491a8154b8fa01ff510e03078ab427960ac017" exitCode=0
Jul 20 22:25:10.551065 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:10.550662    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv" event={"ID":"21d248fd-bc94-4517-86e4-79e38827640e","Type":"ContainerDied","Data":"15674c5073c1e70e25a6bde9dc491a8154b8fa01ff510e03078ab427960ac017"}
Jul 20 22:25:10.551065 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:10.550957    2550 scope.go:117] "RemoveContainer" containerID="15674c5073c1e70e25a6bde9dc491a8154b8fa01ff510e03078ab427960ac017"
Jul 20 22:25:11.555955 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:11.555919    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9kpkv" event={"ID":"21d248fd-bc94-4517-86e4-79e38827640e","Type":"ContainerStarted","Data":"ea520439d420807ae36ef382c91914d3f81a50156045510e7221dfdf287c9c34"}
Jul 20 22:25:13.201336 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:13.201302    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:13.367217 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:13.367185    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:28.038394 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:28.038355    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:28.096844 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:28.096816    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:32.995880 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:32.995846    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:42.796820 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:42.796745    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:42.894536 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:42.894504    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:48.340759 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:48.340706    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:48.441405 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:48.441377    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:52.886582 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:52.886546    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:53.028206 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:53.028166    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:53.123279 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:53.123245    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:53.647316 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:53.647280    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:25:53.703606 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:25:53.703575    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:26:03.341789 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:26:03.341757    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:26:03.443203 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:26:03.443175    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:26:03.541213 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:26:03.541172    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:26:03.609215 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:26:03.609144    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:26:39.782409 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:26:39.782377    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:26:40.896000 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:26:40.895959    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:26:40.956356 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:26:40.956327    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:26:58.011898 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:26:58.011864    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:26:58.078493 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:26:58.078463    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:27:01.080366 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:01.080329    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:27:01.187692 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:01.187652    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:27:09.949642 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:09.949559    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:27:10.019798 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:10.019765    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:27:11.731272 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:11.731240    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:27:11.829391 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:11.829360    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:27:13.083424 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:13.083388    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:27:13.183969 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:13.183934    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:27:21.320007 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:21.319969    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:27:44.735846 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:44.735815    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:27:44.741320 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:44.741295    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:27:44.743155 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:44.743131    2550 kubelet.go:1648] "Image garbage collection succeeded"
Jul 20 22:27:53.271872 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:53.271834    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:27:53.449854 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:27:53.449785    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:16.365653 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:16.365584    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:17.970600 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:17.970560    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:21.138403 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:21.138369    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:21.236074 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:21.236038    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:23.193285 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:23.193253    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:23.432617 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:23.432586    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:23.775119 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:23.775082    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:23.979758 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:23.979703    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:25.513422 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:25.513377    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:26.102785 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:26.102751    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:28.128380 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:28.128346    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:28.228880 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:28.228847    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:30.712343 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:30.712312    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:30.799481 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:30.799446    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:34.471420 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:34.471383    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:34.629643 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:34.629609    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:34.750203 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:34.750112    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:39.516256 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:39.516179    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:39.616265 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:39.616230    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:41.649882 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:41.649851    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:58.592648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:58.592574    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:58.751921 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:58.751851    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:59.288712 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.288664    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:59.694830 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.694704    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:59.803685 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.803649    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:28:59.810319 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.810275    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"]
Jul 20 22:28:59.813339 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.813323    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:28:59.815618 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.815587    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 20 22:28:59.815820 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.815785    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5rcrj\""
Jul 20 22:28:59.815820 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.815814    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 20 22:28:59.816052 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.816030    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 20 22:28:59.816092 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.816058    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 20 22:28:59.841053 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.841026    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"]
Jul 20 22:28:59.948921 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.948842    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71554333-2c49-4889-b65b-08b6af2a1c43-manifests\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:28:59.948921 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.948884    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71554333-2c49-4889-b65b-08b6af2a1c43-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:28:59.948921 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.948905    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blf5z\" (UniqueName: \"kubernetes.io/projected/71554333-2c49-4889-b65b-08b6af2a1c43-kube-api-access-blf5z\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:28:59.949131 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.948954    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71554333-2c49-4889-b65b-08b6af2a1c43-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:28:59.949131 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:28:59.948972    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71554333-2c49-4889-b65b-08b6af2a1c43-tmp\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:00.033756 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.033707    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:00.049643 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.049618    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71554333-2c49-4889-b65b-08b6af2a1c43-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:00.049710 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.049652    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blf5z\" (UniqueName: \"kubernetes.io/projected/71554333-2c49-4889-b65b-08b6af2a1c43-kube-api-access-blf5z\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:00.049710 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.049696    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71554333-2c49-4889-b65b-08b6af2a1c43-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:00.049802 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.049735    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71554333-2c49-4889-b65b-08b6af2a1c43-tmp\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:00.049843 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.049798    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71554333-2c49-4889-b65b-08b6af2a1c43-manifests\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:00.050172 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.050152    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71554333-2c49-4889-b65b-08b6af2a1c43-manifests\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:00.051996 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.051974    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71554333-2c49-4889-b65b-08b6af2a1c43-tmp\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:00.052215 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.052199    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71554333-2c49-4889-b65b-08b6af2a1c43-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:00.052281 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.052264    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71554333-2c49-4889-b65b-08b6af2a1c43-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:00.072092 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.072067    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blf5z\" (UniqueName: \"kubernetes.io/projected/71554333-2c49-4889-b65b-08b6af2a1c43-kube-api-access-blf5z\") pod \"opendatahub-operator-controller-manager-6b4989886d-xc2dt\" (UID: \"71554333-2c49-4889-b65b-08b6af2a1c43\") " pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:00.123575 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.123549    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:00.129436 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.129389    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:00.293835 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.293801    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:00.309886 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.309856    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"]
Jul 20 22:29:00.313050 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:29:00.313028    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71554333_2c49_4889_b65b_08b6af2a1c43.slice/crio-24f1576d22db87b0264d820396e71c3c7e3bf864d9ac60fd3f51c87a124b56bf WatchSource:0}: Error finding container 24f1576d22db87b0264d820396e71c3c7e3bf864d9ac60fd3f51c87a124b56bf: Status 404 returned error can't find the container with id 24f1576d22db87b0264d820396e71c3c7e3bf864d9ac60fd3f51c87a124b56bf
Jul 20 22:29:00.314656 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.314639    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 22:29:00.345645 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.345611    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt" event={"ID":"71554333-2c49-4889-b65b-08b6af2a1c43","Type":"ContainerStarted","Data":"24f1576d22db87b0264d820396e71c3c7e3bf864d9ac60fd3f51c87a124b56bf"}
Jul 20 22:29:00.387009 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.386966    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:00.485083 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:00.485009    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:04.366690 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:04.366656    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt" event={"ID":"71554333-2c49-4889-b65b-08b6af2a1c43","Type":"ContainerStarted","Data":"5ed502d9a76d78867cdd7ae3e59c848fc72d535bfe4aba4edad59fcae4bd436d"}
Jul 20 22:29:04.526746 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:04.526690    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:04.716981 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:04.716957    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:05.371386 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:05.371351    2550 generic.go:356] "Generic (PLEG): container finished" podID="71554333-2c49-4889-b65b-08b6af2a1c43" containerID="5ed502d9a76d78867cdd7ae3e59c848fc72d535bfe4aba4edad59fcae4bd436d" exitCode=0
Jul 20 22:29:05.371754 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:05.371422    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt" event={"ID":"71554333-2c49-4889-b65b-08b6af2a1c43","Type":"ContainerDied","Data":"5ed502d9a76d78867cdd7ae3e59c848fc72d535bfe4aba4edad59fcae4bd436d"}
Jul 20 22:29:06.376991 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:06.376952    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt" event={"ID":"71554333-2c49-4889-b65b-08b6af2a1c43","Type":"ContainerStarted","Data":"4c4d4fbb855ecc6c39669d9dacc6bc6c5d907d7533f1725bd534ab19824877a0"}
Jul 20 22:29:06.377349 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:06.377090    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:06.400190 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:06.400142    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt" podStartSLOduration=2.992029189 podStartE2EDuration="7.400127511s" totalImagesPullingTime="3.327228636s" totalInitContainerRuntime="1.080869686s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:28:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:29:06.39832384 +0000 UTC m=+382.176027892" watchObservedRunningTime="2026-07-20 22:29:06.400127511 +0000 UTC m=+382.177831563"
Jul 20 22:29:10.297979 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:10.297948    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:10.447530 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:10.447498    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:11.230242 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:11.230203    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:11.333637 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:11.333601    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:17.382975 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:17.382941    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b4989886d-xc2dt"
Jul 20 22:29:19.258628 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:19.258590    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:19.393171 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:19.393139    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:20.120286 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.120251    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-q56kq"]
Jul 20 22:29:20.128286 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.128266    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-q56kq"
Jul 20 22:29:20.132930 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.132909    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 20 22:29:20.133029 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.133011    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-222fg\""
Jul 20 22:29:20.152308 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.152285    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-q56kq"]
Jul 20 22:29:20.193845 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.193811    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58ae02e3-3d12-451f-b178-5f7ea84f259c-cert\") pod \"odh-model-controller-7998c67dc4-q56kq\" (UID: \"58ae02e3-3d12-451f-b178-5f7ea84f259c\") " pod="opendatahub/odh-model-controller-7998c67dc4-q56kq"
Jul 20 22:29:20.193977 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.193890    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xr47\" (UniqueName: \"kubernetes.io/projected/58ae02e3-3d12-451f-b178-5f7ea84f259c-kube-api-access-6xr47\") pod \"odh-model-controller-7998c67dc4-q56kq\" (UID: \"58ae02e3-3d12-451f-b178-5f7ea84f259c\") " pod="opendatahub/odh-model-controller-7998c67dc4-q56kq"
Jul 20 22:29:20.209355 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.209327    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:20.295110 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.295084    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xr47\" (UniqueName: \"kubernetes.io/projected/58ae02e3-3d12-451f-b178-5f7ea84f259c-kube-api-access-6xr47\") pod \"odh-model-controller-7998c67dc4-q56kq\" (UID: \"58ae02e3-3d12-451f-b178-5f7ea84f259c\") " pod="opendatahub/odh-model-controller-7998c67dc4-q56kq"
Jul 20 22:29:20.295533 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.295133    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58ae02e3-3d12-451f-b178-5f7ea84f259c-cert\") pod \"odh-model-controller-7998c67dc4-q56kq\" (UID: \"58ae02e3-3d12-451f-b178-5f7ea84f259c\") " pod="opendatahub/odh-model-controller-7998c67dc4-q56kq"
Jul 20 22:29:20.295533 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:29:20.295246    2550 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 20 22:29:20.295533 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:29:20.295298    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ae02e3-3d12-451f-b178-5f7ea84f259c-cert podName:58ae02e3-3d12-451f-b178-5f7ea84f259c nodeName:}" failed. No retries permitted until 2026-07-20 22:29:20.795282854 +0000 UTC m=+396.572986884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/58ae02e3-3d12-451f-b178-5f7ea84f259c-cert") pod "odh-model-controller-7998c67dc4-q56kq" (UID: "58ae02e3-3d12-451f-b178-5f7ea84f259c") : secret "odh-model-controller-webhook-cert" not found
Jul 20 22:29:20.307023 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.306997    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xr47\" (UniqueName: \"kubernetes.io/projected/58ae02e3-3d12-451f-b178-5f7ea84f259c-kube-api-access-6xr47\") pod \"odh-model-controller-7998c67dc4-q56kq\" (UID: \"58ae02e3-3d12-451f-b178-5f7ea84f259c\") " pod="opendatahub/odh-model-controller-7998c67dc4-q56kq"
Jul 20 22:29:20.315548 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.315530    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:20.410217 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.410150    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:20.627517 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.627489    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:20.799578 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.799548    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58ae02e3-3d12-451f-b178-5f7ea84f259c-cert\") pod \"odh-model-controller-7998c67dc4-q56kq\" (UID: \"58ae02e3-3d12-451f-b178-5f7ea84f259c\") " pod="opendatahub/odh-model-controller-7998c67dc4-q56kq"
Jul 20 22:29:20.801853 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.801835    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58ae02e3-3d12-451f-b178-5f7ea84f259c-cert\") pod \"odh-model-controller-7998c67dc4-q56kq\" (UID: \"58ae02e3-3d12-451f-b178-5f7ea84f259c\") " pod="opendatahub/odh-model-controller-7998c67dc4-q56kq"
Jul 20 22:29:20.818129 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.818107    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:20.931852 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:20.931820    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:21.037802 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:21.037768    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-q56kq"
Jul 20 22:29:21.212919 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:21.212890    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-q56kq"]
Jul 20 22:29:21.215709 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:29:21.215683    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ae02e3_3d12_451f_b178_5f7ea84f259c.slice/crio-c06c355399e53595e08aa67e62493a74b9df6cdb2047ab2c00492fde7952960a WatchSource:0}: Error finding container c06c355399e53595e08aa67e62493a74b9df6cdb2047ab2c00492fde7952960a: Status 404 returned error can't find the container with id c06c355399e53595e08aa67e62493a74b9df6cdb2047ab2c00492fde7952960a
Jul 20 22:29:21.429553 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:21.429480    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-q56kq" event={"ID":"58ae02e3-3d12-451f-b178-5f7ea84f259c","Type":"ContainerStarted","Data":"c06c355399e53595e08aa67e62493a74b9df6cdb2047ab2c00492fde7952960a"}
Jul 20 22:29:24.443290 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:24.443250    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-q56kq" event={"ID":"58ae02e3-3d12-451f-b178-5f7ea84f259c","Type":"ContainerStarted","Data":"c50931542a4287ff699ba12e4ed36ed4d07b9e97f3415b9de4b2fd6608058c6d"}
Jul 20 22:29:24.443667 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:24.443368    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-q56kq"
Jul 20 22:29:24.466602 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:24.466554    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-q56kq" podStartSLOduration=1.825780645 podStartE2EDuration="4.46653899s" totalImagesPullingTime="2.640758345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:29:24.464937363 +0000 UTC m=+400.242641417" watchObservedRunningTime="2026-07-20 22:29:24.46653899 +0000 UTC m=+400.244243043"
Jul 20 22:29:30.735514 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.735480    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:30.751527 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.751495    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"]
Jul 20 22:29:30.754746 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.754730    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"
Jul 20 22:29:30.759318 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.759298    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 20 22:29:30.759411 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.759304    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 20 22:29:30.759950 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.759925    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-m2wht\""
Jul 20 22:29:30.781539 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.781520    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"]
Jul 20 22:29:30.880644 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.880620    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t28b6\" (UniqueName: \"kubernetes.io/projected/3b6f9220-0c28-42f7-9d81-9d0eadcf9958-kube-api-access-t28b6\") pod \"kube-auth-proxy-76f85576d5-mqcrz\" (UID: \"3b6f9220-0c28-42f7-9d81-9d0eadcf9958\") " pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"
Jul 20 22:29:30.880773 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.880671    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b6f9220-0c28-42f7-9d81-9d0eadcf9958-tmp\") pod \"kube-auth-proxy-76f85576d5-mqcrz\" (UID: \"3b6f9220-0c28-42f7-9d81-9d0eadcf9958\") " pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"
Jul 20 22:29:30.880773 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.880690    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b6f9220-0c28-42f7-9d81-9d0eadcf9958-tls-certs\") pod \"kube-auth-proxy-76f85576d5-mqcrz\" (UID: \"3b6f9220-0c28-42f7-9d81-9d0eadcf9958\") " pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"
Jul 20 22:29:30.982015 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.981989    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b6f9220-0c28-42f7-9d81-9d0eadcf9958-tls-certs\") pod \"kube-auth-proxy-76f85576d5-mqcrz\" (UID: \"3b6f9220-0c28-42f7-9d81-9d0eadcf9958\") " pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"
Jul 20 22:29:30.982105 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.982035    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t28b6\" (UniqueName: \"kubernetes.io/projected/3b6f9220-0c28-42f7-9d81-9d0eadcf9958-kube-api-access-t28b6\") pod \"kube-auth-proxy-76f85576d5-mqcrz\" (UID: \"3b6f9220-0c28-42f7-9d81-9d0eadcf9958\") " pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"
Jul 20 22:29:30.982143 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.982102    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b6f9220-0c28-42f7-9d81-9d0eadcf9958-tmp\") pod \"kube-auth-proxy-76f85576d5-mqcrz\" (UID: \"3b6f9220-0c28-42f7-9d81-9d0eadcf9958\") " pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"
Jul 20 22:29:30.982179 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:29:30.982148    2550 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 20 22:29:30.982231 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:29:30.982220    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6f9220-0c28-42f7-9d81-9d0eadcf9958-tls-certs podName:3b6f9220-0c28-42f7-9d81-9d0eadcf9958 nodeName:}" failed. No retries permitted until 2026-07-20 22:29:31.482199912 +0000 UTC m=+407.259903957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/3b6f9220-0c28-42f7-9d81-9d0eadcf9958-tls-certs") pod "kube-auth-proxy-76f85576d5-mqcrz" (UID: "3b6f9220-0c28-42f7-9d81-9d0eadcf9958") : secret "kube-auth-proxy-tls" not found
Jul 20 22:29:30.984274 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:30.984246    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b6f9220-0c28-42f7-9d81-9d0eadcf9958-tmp\") pod \"kube-auth-proxy-76f85576d5-mqcrz\" (UID: \"3b6f9220-0c28-42f7-9d81-9d0eadcf9958\") " pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"
Jul 20 22:29:31.099306 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.099251    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t28b6\" (UniqueName: \"kubernetes.io/projected/3b6f9220-0c28-42f7-9d81-9d0eadcf9958-kube-api-access-t28b6\") pod \"kube-auth-proxy-76f85576d5-mqcrz\" (UID: \"3b6f9220-0c28-42f7-9d81-9d0eadcf9958\") " pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"
Jul 20 22:29:31.254349 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.254319    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:31.474055 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.469543    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:31.486703 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.486674    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b6f9220-0c28-42f7-9d81-9d0eadcf9958-tls-certs\") pod \"kube-auth-proxy-76f85576d5-mqcrz\" (UID: \"3b6f9220-0c28-42f7-9d81-9d0eadcf9958\") " pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"
Jul 20 22:29:31.489320 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.489287    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b6f9220-0c28-42f7-9d81-9d0eadcf9958-tls-certs\") pod \"kube-auth-proxy-76f85576d5-mqcrz\" (UID: \"3b6f9220-0c28-42f7-9d81-9d0eadcf9958\") " pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"
Jul 20 22:29:31.496314 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.496290    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx"]
Jul 20 22:29:31.499567 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.499553    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx"
Jul 20 22:29:31.502463 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.502440    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-fkcnx\""
Jul 20 22:29:31.502865 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.502844    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 20 22:29:31.552787 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.552761    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"]
Jul 20 22:29:31.555998 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.555979    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"
Jul 20 22:29:31.560077 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.560056    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 20 22:29:31.560167 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.560101    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6lq29\""
Jul 20 22:29:31.587615 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.587586    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddddd\" (UniqueName: \"kubernetes.io/projected/6f2c38fb-6cc5-438b-b824-a243ca239ad1-kube-api-access-ddddd\") pod \"llmisvc-controller-manager-777c6bd7d5-vk7tj\" (UID: \"6f2c38fb-6cc5-438b-b824-a243ca239ad1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"
Jul 20 22:29:31.587695 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.587625    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/417e7a1c-68f2-4f80-8eca-0b89c8bf3186-cert\") pod \"kserve-controller-manager-d6f4b65b8-f4fmx\" (UID: \"417e7a1c-68f2-4f80-8eca-0b89c8bf3186\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx"
Jul 20 22:29:31.587695 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.587648    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f2c38fb-6cc5-438b-b824-a243ca239ad1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vk7tj\" (UID: \"6f2c38fb-6cc5-438b-b824-a243ca239ad1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"
Jul 20 22:29:31.587803 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.587701    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgd8w\" (UniqueName: \"kubernetes.io/projected/417e7a1c-68f2-4f80-8eca-0b89c8bf3186-kube-api-access-bgd8w\") pod \"kserve-controller-manager-d6f4b65b8-f4fmx\" (UID: \"417e7a1c-68f2-4f80-8eca-0b89c8bf3186\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx"
Jul 20 22:29:31.606233 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.606206    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx"]
Jul 20 22:29:31.631047 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.631022    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"]
Jul 20 22:29:31.644363 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.644343    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:31.664394 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.664368    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"
Jul 20 22:29:31.688468 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.688438    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/417e7a1c-68f2-4f80-8eca-0b89c8bf3186-cert\") pod \"kserve-controller-manager-d6f4b65b8-f4fmx\" (UID: \"417e7a1c-68f2-4f80-8eca-0b89c8bf3186\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx"
Jul 20 22:29:31.688557 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.688476    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f2c38fb-6cc5-438b-b824-a243ca239ad1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vk7tj\" (UID: \"6f2c38fb-6cc5-438b-b824-a243ca239ad1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"
Jul 20 22:29:31.688557 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.688499    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgd8w\" (UniqueName: \"kubernetes.io/projected/417e7a1c-68f2-4f80-8eca-0b89c8bf3186-kube-api-access-bgd8w\") pod \"kserve-controller-manager-d6f4b65b8-f4fmx\" (UID: \"417e7a1c-68f2-4f80-8eca-0b89c8bf3186\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx"
Jul 20 22:29:31.688638 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.688562    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddddd\" (UniqueName: \"kubernetes.io/projected/6f2c38fb-6cc5-438b-b824-a243ca239ad1-kube-api-access-ddddd\") pod \"llmisvc-controller-manager-777c6bd7d5-vk7tj\" (UID: \"6f2c38fb-6cc5-438b-b824-a243ca239ad1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"
Jul 20 22:29:31.688638 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:29:31.688618    2550 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 20 22:29:31.688732 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:29:31.688688    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f2c38fb-6cc5-438b-b824-a243ca239ad1-cert podName:6f2c38fb-6cc5-438b-b824-a243ca239ad1 nodeName:}" failed. No retries permitted until 2026-07-20 22:29:32.188672623 +0000 UTC m=+407.966376653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6f2c38fb-6cc5-438b-b824-a243ca239ad1-cert") pod "llmisvc-controller-manager-777c6bd7d5-vk7tj" (UID: "6f2c38fb-6cc5-438b-b824-a243ca239ad1") : secret "llmisvc-webhook-server-cert" not found
Jul 20 22:29:31.690888 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.690866    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/417e7a1c-68f2-4f80-8eca-0b89c8bf3186-cert\") pod \"kserve-controller-manager-d6f4b65b8-f4fmx\" (UID: \"417e7a1c-68f2-4f80-8eca-0b89c8bf3186\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx"
Jul 20 22:29:31.712829 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.712795    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddddd\" (UniqueName: \"kubernetes.io/projected/6f2c38fb-6cc5-438b-b824-a243ca239ad1-kube-api-access-ddddd\") pod \"llmisvc-controller-manager-777c6bd7d5-vk7tj\" (UID: \"6f2c38fb-6cc5-438b-b824-a243ca239ad1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"
Jul 20 22:29:31.714648 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.714625    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgd8w\" (UniqueName: \"kubernetes.io/projected/417e7a1c-68f2-4f80-8eca-0b89c8bf3186-kube-api-access-bgd8w\") pod \"kserve-controller-manager-d6f4b65b8-f4fmx\" (UID: \"417e7a1c-68f2-4f80-8eca-0b89c8bf3186\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx"
Jul 20 22:29:31.810060 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.810002    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx"
Jul 20 22:29:31.811473 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.811451    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz"]
Jul 20 22:29:31.813806 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:29:31.813778    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b6f9220_0c28_42f7_9d81_9d0eadcf9958.slice/crio-2cc9304e0dcc5c7e887a1674e36d3779c21302e370431c3ed32764148490f5d4 WatchSource:0}: Error finding container 2cc9304e0dcc5c7e887a1674e36d3779c21302e370431c3ed32764148490f5d4: Status 404 returned error can't find the container with id 2cc9304e0dcc5c7e887a1674e36d3779c21302e370431c3ed32764148490f5d4
Jul 20 22:29:31.898920 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.898886    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:31.959136 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:31.959106    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx"]
Jul 20 22:29:31.959993 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:29:31.959968    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod417e7a1c_68f2_4f80_8eca_0b89c8bf3186.slice/crio-e3cae9b003cfda75d1c2478510f942339a905ba42c25128b36073e8a8adb612d WatchSource:0}: Error finding container e3cae9b003cfda75d1c2478510f942339a905ba42c25128b36073e8a8adb612d: Status 404 returned error can't find the container with id e3cae9b003cfda75d1c2478510f942339a905ba42c25128b36073e8a8adb612d
Jul 20 22:29:32.191920 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:32.191838    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f2c38fb-6cc5-438b-b824-a243ca239ad1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vk7tj\" (UID: \"6f2c38fb-6cc5-438b-b824-a243ca239ad1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"
Jul 20 22:29:32.194238 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:32.194221    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f2c38fb-6cc5-438b-b824-a243ca239ad1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vk7tj\" (UID: \"6f2c38fb-6cc5-438b-b824-a243ca239ad1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"
Jul 20 22:29:32.464569 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:32.464539    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"
Jul 20 22:29:32.472516 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:32.472490    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx" event={"ID":"417e7a1c-68f2-4f80-8eca-0b89c8bf3186","Type":"ContainerStarted","Data":"e3cae9b003cfda75d1c2478510f942339a905ba42c25128b36073e8a8adb612d"}
Jul 20 22:29:32.473610 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:32.473587    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz" event={"ID":"3b6f9220-0c28-42f7-9d81-9d0eadcf9958","Type":"ContainerStarted","Data":"2cc9304e0dcc5c7e887a1674e36d3779c21302e370431c3ed32764148490f5d4"}
Jul 20 22:29:32.696203 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:32.696167    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"]
Jul 20 22:29:32.700293 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:29:32.700261    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6f2c38fb_6cc5_438b_b824_a243ca239ad1.slice/crio-51664e95127e7e6f4a2b433091c8693e9018d32a7486ad5860139877ca6e9609 WatchSource:0}: Error finding container 51664e95127e7e6f4a2b433091c8693e9018d32a7486ad5860139877ca6e9609: Status 404 returned error can't find the container with id 51664e95127e7e6f4a2b433091c8693e9018d32a7486ad5860139877ca6e9609
Jul 20 22:29:32.943329 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:32.943193    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:33.079822 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:33.079794    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:33.249892 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:33.249854    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:33.479842 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:33.479698    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj" event={"ID":"6f2c38fb-6cc5-438b-b824-a243ca239ad1","Type":"ContainerStarted","Data":"51664e95127e7e6f4a2b433091c8693e9018d32a7486ad5860139877ca6e9609"}
Jul 20 22:29:35.449631 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:35.449601    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-q56kq"
Jul 20 22:29:36.494013 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.493975    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx" event={"ID":"417e7a1c-68f2-4f80-8eca-0b89c8bf3186","Type":"ContainerStarted","Data":"1e8b8b951c9375f60dd47d289455bca4d52baf83c862497202436ebffdd2ccda"}
Jul 20 22:29:36.494462 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.494126    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx"
Jul 20 22:29:36.495483 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.495455    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz" event={"ID":"3b6f9220-0c28-42f7-9d81-9d0eadcf9958","Type":"ContainerStarted","Data":"73f82cd3f49da1ccce3fd78c7dde6d0e26ec6ec5194bad86393509362af51cf3"}
Jul 20 22:29:36.496815 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.496787    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj" event={"ID":"6f2c38fb-6cc5-438b-b824-a243ca239ad1","Type":"ContainerStarted","Data":"bfab73a9a3d7890523e1bfd402f32ff9e06c01e73aefbc68e098aa2783e10415"}
Jul 20 22:29:36.496935 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.496920    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"
Jul 20 22:29:36.510711 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.510690    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:36.816392 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.816298    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx" podStartSLOduration=1.9884007860000001 podStartE2EDuration="5.816280469s" totalImagesPullingTime="3.827879683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:29:36.732511211 +0000 UTC m=+412.510215274" watchObservedRunningTime="2026-07-20 22:29:36.816280469 +0000 UTC m=+412.593984519"
Jul 20 22:29:36.816557 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.816425    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"]
Jul 20 22:29:36.820532 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.820510    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:36.827435 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.827413    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 20 22:29:36.828083 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.828059    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 20 22:29:36.830020 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.829999    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 20 22:29:36.830476 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.830461    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 22:29:36.833677 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.833660    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4nfjp\""
Jul 20 22:29:36.847993 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.847957    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-76f85576d5-mqcrz" podStartSLOduration=2.876916972 podStartE2EDuration="6.847944713s" totalImagesPullingTime="3.971027741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:29:36.846351439 +0000 UTC m=+412.624055492" watchObservedRunningTime="2026-07-20 22:29:36.847944713 +0000 UTC m=+412.625648812"
Jul 20 22:29:36.858387 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.858347    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:36.864928 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.864907    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"]
Jul 20 22:29:36.931166 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.931110    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj" podStartSLOduration=2.779529292 podStartE2EDuration="5.931095495s" totalImagesPullingTime="3.151566203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:29:36.930205667 +0000 UTC m=+412.707909719" watchObservedRunningTime="2026-07-20 22:29:36.931095495 +0000 UTC m=+412.708799570"
Jul 20 22:29:36.934482 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.934456    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e0931d18-25b6-425b-acd7-075c46f058dc-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:36.934559 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.934496    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e0931d18-25b6-425b-acd7-075c46f058dc-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:36.934559 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.934514    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e0931d18-25b6-425b-acd7-075c46f058dc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:36.934679 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.934561    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e0931d18-25b6-425b-acd7-075c46f058dc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:36.934679 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.934624    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0931d18-25b6-425b-acd7-075c46f058dc-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:36.934679 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.934654    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swnpd\" (UniqueName: \"kubernetes.io/projected/e0931d18-25b6-425b-acd7-075c46f058dc-kube-api-access-swnpd\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:36.934856 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:36.934769    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0931d18-25b6-425b-acd7-075c46f058dc-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.035979 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.035947    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0931d18-25b6-425b-acd7-075c46f058dc-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.036152 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.035987    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e0931d18-25b6-425b-acd7-075c46f058dc-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.036152 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.036019    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e0931d18-25b6-425b-acd7-075c46f058dc-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.036152 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.036046    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e0931d18-25b6-425b-acd7-075c46f058dc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.036315 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.036192    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e0931d18-25b6-425b-acd7-075c46f058dc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.036879 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.036855    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0931d18-25b6-425b-acd7-075c46f058dc-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.036981 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.036895    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swnpd\" (UniqueName: \"kubernetes.io/projected/e0931d18-25b6-425b-acd7-075c46f058dc-kube-api-access-swnpd\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.037055 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.036855    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e0931d18-25b6-425b-acd7-075c46f058dc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.038635 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.038607    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e0931d18-25b6-425b-acd7-075c46f058dc-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.038635 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.038615    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e0931d18-25b6-425b-acd7-075c46f058dc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.038952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.038932    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e0931d18-25b6-425b-acd7-075c46f058dc-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.039035 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.039017    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0931d18-25b6-425b-acd7-075c46f058dc-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.057875 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.057856    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0931d18-25b6-425b-acd7-075c46f058dc-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.075492 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.075434    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swnpd\" (UniqueName: \"kubernetes.io/projected/e0931d18-25b6-425b-acd7-075c46f058dc-kube-api-access-swnpd\") pod \"istiod-openshift-gateway-94bb8fbfd-ltpmz\" (UID: \"e0931d18-25b6-425b-acd7-075c46f058dc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.095972 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.095951    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:37.129964 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.129941    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:37.284310 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:29:37.284273    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0931d18_25b6_425b_acd7_075c46f058dc.slice/crio-1d6a57bd398c03a8aa0a63ce69d3fccc4e9615e7ad4f291aa6ecea1c2b7677d2 WatchSource:0}: Error finding container 1d6a57bd398c03a8aa0a63ce69d3fccc4e9615e7ad4f291aa6ecea1c2b7677d2: Status 404 returned error can't find the container with id 1d6a57bd398c03a8aa0a63ce69d3fccc4e9615e7ad4f291aa6ecea1c2b7677d2
Jul 20 22:29:37.284672 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.284648    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"]
Jul 20 22:29:37.502349 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:37.502315    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz" event={"ID":"e0931d18-25b6-425b-acd7-075c46f058dc","Type":"ContainerStarted","Data":"1d6a57bd398c03a8aa0a63ce69d3fccc4e9615e7ad4f291aa6ecea1c2b7677d2"}
Jul 20 22:29:39.774607 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:39.774556    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 22:29:39.774923 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:39.774645    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 22:29:40.360323 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:40.360291    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:40.489332 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:40.489299    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:40.520826 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:40.520791    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz" event={"ID":"e0931d18-25b6-425b-acd7-075c46f058dc","Type":"ContainerStarted","Data":"cd83f6c5cb981a28bf561948317d7aa115aa64796499d085f8d15442d6c6d408"}
Jul 20 22:29:40.521029 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:40.521000    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:40.522649 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:40.522624    2550 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-ltpmz container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 22:29:40.522770 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:40.522671    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz" podUID="e0931d18-25b6-425b-acd7-075c46f058dc" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 22:29:40.552635 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:40.552591    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz" podStartSLOduration=2.064443415 podStartE2EDuration="4.552578956s" totalImagesPullingTime="2.488135541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:29:40.550444713 +0000 UTC m=+416.328148769" watchObservedRunningTime="2026-07-20 22:29:40.552578956 +0000 UTC m=+416.330283007"
Jul 20 22:29:41.145578 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:41.145542    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:41.465864 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:41.465828    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:41.525129 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:41.525110    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ltpmz"
Jul 20 22:29:41.675079 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:41.675039    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:29:44.866132 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:29:44.866094    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:07.504930 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:07.504860    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vk7tj"
Jul 20 22:30:07.508392 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:07.508368    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4fmx"
Jul 20 22:30:20.703680 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:20.703643    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:20.796915 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:20.796883    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:20.880462 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:20.880425    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:45.086987 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:45.086952    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:45.839672 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:45.839639    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:50.486275 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:50.486242    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:51.847496 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:51.847466    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:52.141082 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:52.141007    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:53.190624 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:53.190585    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt"]
Jul 20 22:30:53.192880 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:53.192859    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt"
Jul 20 22:30:53.196695 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:53.196674    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 20 22:30:53.197568 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:53.197549    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 20 22:30:53.207485 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:30:53.207454    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"dns-operator-controller-manager-864dd78c4c-ldbtt\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="02e0c2f7-9618-4c6f-bfb8-cec0553c0c9c" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt"
Jul 20 22:30:53.207553 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:30:53.207462    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"dns-operator-controller-env\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" type="*v1.ConfigMap"
Jul 20 22:30:53.207553 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:30:53.207462    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"dns-operator-controller-manager-dockercfg-k8nn6\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-k8nn6\"" type="*v1.Secret"
Jul 20 22:30:53.222274 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:53.222253    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt"]
Jul 20 22:30:53.315029 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:53.315005    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ptx9\" (UniqueName: \"kubernetes.io/projected/02e0c2f7-9618-4c6f-bfb8-cec0553c0c9c-kube-api-access-4ptx9\") pod \"dns-operator-controller-manager-864dd78c4c-ldbtt\" (UID: \"02e0c2f7-9618-4c6f-bfb8-cec0553c0c9c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt"
Jul 20 22:30:53.416374 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:53.416334    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ptx9\" (UniqueName: \"kubernetes.io/projected/02e0c2f7-9618-4c6f-bfb8-cec0553c0c9c-kube-api-access-4ptx9\") pod \"dns-operator-controller-manager-864dd78c4c-ldbtt\" (UID: \"02e0c2f7-9618-4c6f-bfb8-cec0553c0c9c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt"
Jul 20 22:30:53.434069 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:53.434040    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ptx9\" (UniqueName: \"kubernetes.io/projected/02e0c2f7-9618-4c6f-bfb8-cec0553c0c9c-kube-api-access-4ptx9\") pod \"dns-operator-controller-manager-864dd78c4c-ldbtt\" (UID: \"02e0c2f7-9618-4c6f-bfb8-cec0553c0c9c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt"
Jul 20 22:30:53.760234 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:53.760199    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:54.054443 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:54.054368    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:54.176533 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:54.176503    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 20 22:30:54.387679 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:54.387607    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-k8nn6\""
Jul 20 22:30:54.393836 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:54.393821    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt"
Jul 20 22:30:54.543272 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:54.543242    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt"]
Jul 20 22:30:54.544708 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:30:54.544684    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e0c2f7_9618_4c6f_bfb8_cec0553c0c9c.slice/crio-96196fb60adbcd77acbb56d1900cd696cd939f650d9f5ee809b74ce83ddd4cb8 WatchSource:0}: Error finding container 96196fb60adbcd77acbb56d1900cd696cd939f650d9f5ee809b74ce83ddd4cb8: Status 404 returned error can't find the container with id 96196fb60adbcd77acbb56d1900cd696cd939f650d9f5ee809b74ce83ddd4cb8
Jul 20 22:30:54.824856 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:54.824828    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt" event={"ID":"02e0c2f7-9618-4c6f-bfb8-cec0553c0c9c","Type":"ContainerStarted","Data":"96196fb60adbcd77acbb56d1900cd696cd939f650d9f5ee809b74ce83ddd4cb8"}
Jul 20 22:30:56.785640 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:56.785604    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:56.834756 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:56.834712    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt" event={"ID":"02e0c2f7-9618-4c6f-bfb8-cec0553c0c9c","Type":"ContainerStarted","Data":"7b57ead53a5d130aa1adc85884f55c868717debee9bb8518cd7ecb7f6da7dc84"}
Jul 20 22:30:56.834885 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:56.834834    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt"
Jul 20 22:30:56.860410 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:56.860368    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt" podStartSLOduration=2.216158684 podStartE2EDuration="3.860355233s" totalImagesPullingTime="1.644196549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:30:56.860022573 +0000 UTC m=+492.637726607" watchObservedRunningTime="2026-07-20 22:30:56.860355233 +0000 UTC m=+492.638059356"
Jul 20 22:30:57.176208 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.176134    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"]
Jul 20 22:30:57.178763 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.178745    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"
Jul 20 22:30:57.181414 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.181392    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-752vt\""
Jul 20 22:30:57.203128 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.203098    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"]
Jul 20 22:30:57.246646 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.246620    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbwv8\" (UniqueName: \"kubernetes.io/projected/700247d9-9476-4232-ada3-a3f4bfdc5b51-kube-api-access-kbwv8\") pod \"limitador-operator-controller-manager-859d7b78bd-qs8kt\" (UID: \"700247d9-9476-4232-ada3-a3f4bfdc5b51\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"
Jul 20 22:30:57.269646 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.269627    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:57.346965 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.346935    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbwv8\" (UniqueName: \"kubernetes.io/projected/700247d9-9476-4232-ada3-a3f4bfdc5b51-kube-api-access-kbwv8\") pod \"limitador-operator-controller-manager-859d7b78bd-qs8kt\" (UID: \"700247d9-9476-4232-ada3-a3f4bfdc5b51\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"
Jul 20 22:30:57.359974 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.359953    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbwv8\" (UniqueName: \"kubernetes.io/projected/700247d9-9476-4232-ada3-a3f4bfdc5b51-kube-api-access-kbwv8\") pod \"limitador-operator-controller-manager-859d7b78bd-qs8kt\" (UID: \"700247d9-9476-4232-ada3-a3f4bfdc5b51\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"
Jul 20 22:30:57.426005 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.425976    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:57.488995 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.488971    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"
Jul 20 22:30:57.598633 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.598591    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:57.661580 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.661552    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"]
Jul 20 22:30:57.662776 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:30:57.662710    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod700247d9_9476_4232_ada3_a3f4bfdc5b51.slice/crio-f7236c9864514b777f5df2ed6c66193542959dbae38e3e99e7b37130c1d2de77 WatchSource:0}: Error finding container f7236c9864514b777f5df2ed6c66193542959dbae38e3e99e7b37130c1d2de77: Status 404 returned error can't find the container with id f7236c9864514b777f5df2ed6c66193542959dbae38e3e99e7b37130c1d2de77
Jul 20 22:30:57.815007 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.814926    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:57.840125 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.840087    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt" event={"ID":"700247d9-9476-4232-ada3-a3f4bfdc5b51","Type":"ContainerStarted","Data":"f7236c9864514b777f5df2ed6c66193542959dbae38e3e99e7b37130c1d2de77"}
Jul 20 22:30:57.994275 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:57.994240    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:58.461163 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:58.461122    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:58.703394 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:58.703359    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:30:59.851497 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:59.851463    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt" event={"ID":"700247d9-9476-4232-ada3-a3f4bfdc5b51","Type":"ContainerStarted","Data":"417f17d930430ac68b69b207355beca054fc1e180d0900a06fc15a7841ac92cf"}
Jul 20 22:30:59.851874 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:59.851550    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"
Jul 20 22:30:59.878853 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:30:59.878796    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt" podStartSLOduration=1.185262823 podStartE2EDuration="2.878784184s" totalImagesPullingTime="1.693521361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:30:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:30:59.876934626 +0000 UTC m=+495.654638677" watchObservedRunningTime="2026-07-20 22:30:59.878784184 +0000 UTC m=+495.656488235"
Jul 20 22:31:02.403511 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:02.403480    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:02.506031 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:02.505995    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:02.813620 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:02.813588    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:02.954113 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:02.954081    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:05.278507 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:05.278475    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:05.416081 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:05.416047    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:07.842771 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:07.842739    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ldbtt"
Jul 20 22:31:08.756633 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:08.756601    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:08.884870 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:08.884836    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:08.991298 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:08.991268    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:09.445089 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:09.445057    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:09.604497 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:09.604465    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:10.126095 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:10.126056    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:10.857070 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:10.857043    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"
Jul 20 22:31:13.046740 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:13.046687    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:13.147283 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:13.147249    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:20.515460 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.515423    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"]
Jul 20 22:31:20.515892 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.515652    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt" podUID="700247d9-9476-4232-ada3-a3f4bfdc5b51" containerName="manager" containerID="cri-o://417f17d930430ac68b69b207355beca054fc1e180d0900a06fc15a7841ac92cf" gracePeriod=2
Jul 20 22:31:20.531929 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.531900    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"]
Jul 20 22:31:20.538484 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:31:20.538425    2550 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"700247d9-9476-4232-ada3-a3f4bfdc5b51\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-qs8kt\": pods \"limitador-operator-controller-manager-859d7b78bd-qs8kt\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"
Jul 20 22:31:20.559561 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.559529    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6fq76"]
Jul 20 22:31:20.560068 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.560047    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="700247d9-9476-4232-ada3-a3f4bfdc5b51" containerName="manager"
Jul 20 22:31:20.560121 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.560072    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="700247d9-9476-4232-ada3-a3f4bfdc5b51" containerName="manager"
Jul 20 22:31:20.560195 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.560183    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="700247d9-9476-4232-ada3-a3f4bfdc5b51" containerName="manager"
Jul 20 22:31:20.563915 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.563896    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6fq76"
Jul 20 22:31:20.566006 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:31:20.565973    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qs8kt\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="700247d9-9476-4232-ada3-a3f4bfdc5b51" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"
Jul 20 22:31:20.575583 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.575546    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6fq76"]
Jul 20 22:31:20.736637 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.736606    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9zk8\" (UniqueName: \"kubernetes.io/projected/8399921c-0a40-4f17-8797-85a780ad4432-kube-api-access-q9zk8\") pod \"limitador-operator-controller-manager-859d7b78bd-6fq76\" (UID: \"8399921c-0a40-4f17-8797-85a780ad4432\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6fq76"
Jul 20 22:31:20.757188 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.757157    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"
Jul 20 22:31:20.759402 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:31:20.759369    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qs8kt\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="700247d9-9476-4232-ada3-a3f4bfdc5b51" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"
Jul 20 22:31:20.837494 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.837406    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9zk8\" (UniqueName: \"kubernetes.io/projected/8399921c-0a40-4f17-8797-85a780ad4432-kube-api-access-q9zk8\") pod \"limitador-operator-controller-manager-859d7b78bd-6fq76\" (UID: \"8399921c-0a40-4f17-8797-85a780ad4432\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6fq76"
Jul 20 22:31:20.847550 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.847519    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9zk8\" (UniqueName: \"kubernetes.io/projected/8399921c-0a40-4f17-8797-85a780ad4432-kube-api-access-q9zk8\") pod \"limitador-operator-controller-manager-859d7b78bd-6fq76\" (UID: \"8399921c-0a40-4f17-8797-85a780ad4432\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6fq76"
Jul 20 22:31:20.894454 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.894415    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6fq76"
Jul 20 22:31:20.937777 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.937753    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbwv8\" (UniqueName: \"kubernetes.io/projected/700247d9-9476-4232-ada3-a3f4bfdc5b51-kube-api-access-kbwv8\") pod \"700247d9-9476-4232-ada3-a3f4bfdc5b51\" (UID: \"700247d9-9476-4232-ada3-a3f4bfdc5b51\") "
Jul 20 22:31:20.938728 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.938684    2550 generic.go:356] "Generic (PLEG): container finished" podID="700247d9-9476-4232-ada3-a3f4bfdc5b51" containerID="417f17d930430ac68b69b207355beca054fc1e180d0900a06fc15a7841ac92cf" exitCode=0
Jul 20 22:31:20.938869 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.938753    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qs8kt"
Jul 20 22:31:20.938869 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.938772    2550 scope.go:117] "RemoveContainer" containerID="417f17d930430ac68b69b207355beca054fc1e180d0900a06fc15a7841ac92cf"
Jul 20 22:31:20.940127 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.940103    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/700247d9-9476-4232-ada3-a3f4bfdc5b51-kube-api-access-kbwv8" (OuterVolumeSpecName: "kube-api-access-kbwv8") pod "700247d9-9476-4232-ada3-a3f4bfdc5b51" (UID: "700247d9-9476-4232-ada3-a3f4bfdc5b51"). InnerVolumeSpecName "kube-api-access-kbwv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 22:31:20.952035 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.951941    2550 scope.go:117] "RemoveContainer" containerID="417f17d930430ac68b69b207355beca054fc1e180d0900a06fc15a7841ac92cf"
Jul 20 22:31:20.952331 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:31:20.952297    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"417f17d930430ac68b69b207355beca054fc1e180d0900a06fc15a7841ac92cf\": container with ID starting with 417f17d930430ac68b69b207355beca054fc1e180d0900a06fc15a7841ac92cf not found: ID does not exist" containerID="417f17d930430ac68b69b207355beca054fc1e180d0900a06fc15a7841ac92cf"
Jul 20 22:31:20.952434 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:20.952335    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"417f17d930430ac68b69b207355beca054fc1e180d0900a06fc15a7841ac92cf"} err="failed to get container status \"417f17d930430ac68b69b207355beca054fc1e180d0900a06fc15a7841ac92cf\": rpc error: code = NotFound desc = could not find container \"417f17d930430ac68b69b207355beca054fc1e180d0900a06fc15a7841ac92cf\": container with ID starting with 417f17d930430ac68b69b207355beca054fc1e180d0900a06fc15a7841ac92cf not found: ID does not exist"
Jul 20 22:31:21.038456 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:21.038423    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbwv8\" (UniqueName: \"kubernetes.io/projected/700247d9-9476-4232-ada3-a3f4bfdc5b51-kube-api-access-kbwv8\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:31:21.055300 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:21.055272    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6fq76"]
Jul 20 22:31:21.057487 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:31:21.057462    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8399921c_0a40_4f17_8797_85a780ad4432.slice/crio-908991df01d9a801720e54b132851c0fa5c64237bb3b9937439147441f53b705 WatchSource:0}: Error finding container 908991df01d9a801720e54b132851c0fa5c64237bb3b9937439147441f53b705: Status 404 returned error can't find the container with id 908991df01d9a801720e54b132851c0fa5c64237bb3b9937439147441f53b705
Jul 20 22:31:21.946251 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:21.946169    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6fq76" event={"ID":"8399921c-0a40-4f17-8797-85a780ad4432","Type":"ContainerStarted","Data":"1053e909238181ef0459c3fd4877d812dc010492c9cec455b5e9564f937a9299"}
Jul 20 22:31:21.946251 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:21.946226    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6fq76" event={"ID":"8399921c-0a40-4f17-8797-85a780ad4432","Type":"ContainerStarted","Data":"908991df01d9a801720e54b132851c0fa5c64237bb3b9937439147441f53b705"}
Jul 20 22:31:21.946612 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:21.946256    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6fq76"
Jul 20 22:31:21.979556 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:21.979509    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6fq76" podStartSLOduration=1.979494478 podStartE2EDuration="1.979494478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:31:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:31:21.977563522 +0000 UTC m=+517.755267575" watchObservedRunningTime="2026-07-20 22:31:21.979494478 +0000 UTC m=+517.757198529"
Jul 20 22:31:22.844666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:22.844634    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="700247d9-9476-4232-ada3-a3f4bfdc5b51" path="/var/lib/kubelet/pods/700247d9-9476-4232-ada3-a3f4bfdc5b51/volumes"
Jul 20 22:31:31.542136 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:31.542093    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:31.622049 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:31.622018    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:32.951853 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:32.951823    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6fq76"
Jul 20 22:31:50.123432 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:50.123382    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:50.554781 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:50.554748    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:51.872894 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:51.872860    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:52.018282 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.018252    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:52.146531 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.146421    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:52.373038 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.373003    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:31:52.376011 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.375994    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-p9k95"
Jul 20 22:31:52.379990 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.379958    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 20 22:31:52.380136 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.380116    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rtjj7\""
Jul 20 22:31:52.395782 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.395755    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:31:52.468557 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.468532    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:31:52.486804 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.486781    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/195cf9ea-d5a7-4cd9-a08c-405bd180186c-config-file\") pod \"limitador-limitador-54d6d988d5-p9k95\" (UID: \"195cf9ea-d5a7-4cd9-a08c-405bd180186c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p9k95"
Jul 20 22:31:52.486893 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.486815    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btcl4\" (UniqueName: \"kubernetes.io/projected/195cf9ea-d5a7-4cd9-a08c-405bd180186c-kube-api-access-btcl4\") pod \"limitador-limitador-54d6d988d5-p9k95\" (UID: \"195cf9ea-d5a7-4cd9-a08c-405bd180186c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p9k95"
Jul 20 22:31:52.587555 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.587530    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/195cf9ea-d5a7-4cd9-a08c-405bd180186c-config-file\") pod \"limitador-limitador-54d6d988d5-p9k95\" (UID: \"195cf9ea-d5a7-4cd9-a08c-405bd180186c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p9k95"
Jul 20 22:31:52.587663 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.587564    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btcl4\" (UniqueName: \"kubernetes.io/projected/195cf9ea-d5a7-4cd9-a08c-405bd180186c-kube-api-access-btcl4\") pod \"limitador-limitador-54d6d988d5-p9k95\" (UID: \"195cf9ea-d5a7-4cd9-a08c-405bd180186c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p9k95"
Jul 20 22:31:52.588182 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.588161    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/195cf9ea-d5a7-4cd9-a08c-405bd180186c-config-file\") pod \"limitador-limitador-54d6d988d5-p9k95\" (UID: \"195cf9ea-d5a7-4cd9-a08c-405bd180186c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p9k95"
Jul 20 22:31:52.596394 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.596373    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btcl4\" (UniqueName: \"kubernetes.io/projected/195cf9ea-d5a7-4cd9-a08c-405bd180186c-kube-api-access-btcl4\") pod \"limitador-limitador-54d6d988d5-p9k95\" (UID: \"195cf9ea-d5a7-4cd9-a08c-405bd180186c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p9k95"
Jul 20 22:31:52.685522 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.685482    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-p9k95"
Jul 20 22:31:52.820787 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.820608    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:31:52.823444 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:31:52.823411    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod195cf9ea_d5a7_4cd9_a08c_405bd180186c.slice/crio-2801e9a612f5e490c3b3aca19ac6db18a3b078a1608f76d33c12ed6dbaabd4f2 WatchSource:0}: Error finding container 2801e9a612f5e490c3b3aca19ac6db18a3b078a1608f76d33c12ed6dbaabd4f2: Status 404 returned error can't find the container with id 2801e9a612f5e490c3b3aca19ac6db18a3b078a1608f76d33c12ed6dbaabd4f2
Jul 20 22:31:52.922068 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:52.922035    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:53.029047 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:53.028962    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:31:53.078901 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:53.078867    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-p9k95" event={"ID":"195cf9ea-d5a7-4cd9-a08c-405bd180186c","Type":"ContainerStarted","Data":"2801e9a612f5e490c3b3aca19ac6db18a3b078a1608f76d33c12ed6dbaabd4f2"}
Jul 20 22:31:56.094734 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:56.094694    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-p9k95" event={"ID":"195cf9ea-d5a7-4cd9-a08c-405bd180186c","Type":"ContainerStarted","Data":"5c4cf65982fb944ed416f4327d82291ab1aaeaccdbabeb1618bcbf0035a7dd03"}
Jul 20 22:31:56.095134 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:56.094842    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-p9k95"
Jul 20 22:31:56.112464 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:31:56.112412    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-p9k95" podStartSLOduration=1.829779389 podStartE2EDuration="4.112398537s" totalImagesPullingTime="2.282619148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:31:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:31:56.11128644 +0000 UTC m=+551.888990587" watchObservedRunningTime="2026-07-20 22:31:56.112398537 +0000 UTC m=+551.890102589"
Jul 20 22:32:07.100898 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:07.100867    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-p9k95"
Jul 20 22:32:13.043509 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:13.043464    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:32:13.130981 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:13.130950    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:32:13.561854 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:13.561819    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:32:26.303192 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:26.303159    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:32:26.678035 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:26.677956    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:32:32.678232 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:32.678194    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:32:32.760184 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:32.760151    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:32:32.831774 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:32.831744    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:32:32.966888 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:32.966847    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:32:34.762694 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:34.762657    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:32:34.816664 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:34.816632    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:32:35.938083 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:35.938036    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:32:36.083183 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:36.083149    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:32:44.769966 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:44.769938    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:32:44.777763 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:32:44.777737    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:33:05.241226 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:05.241195    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:33:05.332764 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:05.332711    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:33:05.768624 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:05.768596    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:33:06.089398 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:06.089317    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:33:06.487102 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:06.487072    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:33:06.579854 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:06.579820    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:33:06.749827 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:06.749729    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:33:16.125856 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:16.125823    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:33:16.224700 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:16.224655    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:33:27.444436 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:27.444386    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:33:27.936666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:27.936629    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:33:31.635617 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:31.635582    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:33:32.478622 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:32.478585    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:33:51.316918 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:51.316846    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:33:52.373799 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:33:52.373755    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:34:04.184816 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:04.184780    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:34:07.036605 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:07.036556    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:34:08.590080 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:08.590041    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:34:21.597665 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.597630    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"]
Jul 20 22:34:21.601169 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.601145    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.603635 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.603604    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 20 22:34:21.604385 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.604367    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z682z\""
Jul 20 22:34:21.604471 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.604405    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 20 22:34:21.604471 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.604452    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 20 22:34:21.611524 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.611501    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"]
Jul 20 22:34:21.648836 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.648795    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:34:21.685050 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.685007    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.685050 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.685054    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.685299 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.685082    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.685299 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.685172    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.685299 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.685211    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.685299 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.685254    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kjdw\" (UniqueName: \"kubernetes.io/projected/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-kube-api-access-5kjdw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.786048 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.786007    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.786048 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.786047    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.786259 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.786075    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.786259 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.786093    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.786259 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.786192    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kjdw\" (UniqueName: \"kubernetes.io/projected/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-kube-api-access-5kjdw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.786365 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.786260    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.786554 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.786532    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.786617 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.786546    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.786617 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.786591    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.788389 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.788364    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.788642 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.788626    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.797394 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.797374    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kjdw\" (UniqueName: \"kubernetes.io/projected/b0d5a181-bf1f-47af-929e-cdab9a8d0d80-kube-api-access-5kjdw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9wq8\" (UID: \"b0d5a181-bf1f-47af-929e-cdab9a8d0d80\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:21.915186 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:21.915092    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:22.055147 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:22.055120    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"]
Jul 20 22:34:22.057575 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:34:22.057542    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0d5a181_bf1f_47af_929e_cdab9a8d0d80.slice/crio-a8ec1fc1178e8ee8aa1688796cda1db079b8a2b174efa78b4b60202f9c36c6a3 WatchSource:0}: Error finding container a8ec1fc1178e8ee8aa1688796cda1db079b8a2b174efa78b4b60202f9c36c6a3: Status 404 returned error can't find the container with id a8ec1fc1178e8ee8aa1688796cda1db079b8a2b174efa78b4b60202f9c36c6a3
Jul 20 22:34:22.058922 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:22.058904    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 22:34:22.722761 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:22.722702    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8" event={"ID":"b0d5a181-bf1f-47af-929e-cdab9a8d0d80","Type":"ContainerStarted","Data":"a8ec1fc1178e8ee8aa1688796cda1db079b8a2b174efa78b4b60202f9c36c6a3"}
Jul 20 22:34:23.681756 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:23.681705    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:34:23.728034 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:23.727991    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8" event={"ID":"b0d5a181-bf1f-47af-929e-cdab9a8d0d80","Type":"ContainerStarted","Data":"50b0bc95c06b20eacdf67e613c36108b09296901d4d478bb79dd7bf07ad412ec"}
Jul 20 22:34:23.746711 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:23.746658    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8" podStartSLOduration=1.738527087 podStartE2EDuration="2.746641306s" totalImagesPullingTime="1.008114219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:34:23.746522627 +0000 UTC m=+699.524226680" watchObservedRunningTime="2026-07-20 22:34:23.746641306 +0000 UTC m=+699.524345359"
Jul 20 22:34:25.649844 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:25.649806    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:34:31.915390 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:31.915358    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:31.915779 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:31.915420    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:31.927827 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:31.927805    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:32.780045 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:32.780014    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9wq8"
Jul 20 22:34:35.872615 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:35.872580    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:34:39.104734 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:34:39.104629    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:35:00.887945 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:35:00.887908    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:35:38.219929 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:35:38.219896    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:35:38.322473 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:35:38.322439    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:35:39.334464 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:35:39.334426    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:35:39.497474 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:35:39.497443    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:36:00.894492 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:36:00.894450    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:36:00.973322 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:36:00.973290    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:36:11.182619 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:36:11.182533    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:36:20.167918 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:36:20.167882    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:36:31.171590 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:36:31.171548    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:37:44.804777 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:37:44.804675    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:37:44.805265 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:37:44.804995    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:37:47.227883 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:37:47.227850    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:37:48.409502 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:37:48.409463    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:38:13.082018 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:38:13.081987    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:38:20.666337 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:38:20.666300    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:38:23.997143 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:38:23.997112    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:38:24.098125 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:38:24.098096    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:38:24.214334 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:38:24.214294    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:38:30.076858 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:38:30.076816    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:38:31.187110 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:38:31.187076    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:38:36.377118 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:38:36.377075    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:38:36.478303 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:38:36.478274    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:38:46.328341 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:38:46.328304    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:38:54.371104 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:38:54.371064    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:39:03.863051 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:39:03.863012    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:39:13.972661 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:39:13.972586    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:39:21.766058 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:39:21.766015    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:39:28.294567 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:39:28.294531    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:39:28.394649 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:39:28.394605    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:39:33.377767 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:39:33.377707    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:39:45.769784 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:39:45.769746    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:39:56.772984 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:39:56.772941    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:40:06.972002 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:40:06.971966    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:40:28.365055 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:40:28.365021    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:40:37.869857 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:40:37.869780    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:41:02.384485 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:41:02.384447    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:41:13.763732 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:41:13.763692    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:41:29.972133 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:41:29.972097    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:41:39.369385 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:41:39.369352    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:41:49.681902 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:41:49.681870    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:41:59.668779 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:41:59.668738    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:42:09.487426 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:42:09.487332    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:42:29.785860 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:42:29.785826    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:42:44.831692 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:42:44.831665    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:42:44.832389 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:42:44.832370    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:43:57.073338 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:43:57.073262    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:44:00.296496 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:44:00.296460    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:44:08.667628 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:44:08.667586    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:44:17.783343 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:44:17.783309    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:44:29.973975 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:44:29.973943    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:44:41.999324 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:44:41.999284    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:44:51.982821 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:44:51.982790    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:45:00.577499 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:45:00.577464    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:45:10.480686 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:45:10.480607    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:45:22.781858 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:45:22.781826    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:45:31.473800 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:45:31.473767    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:45:41.478892 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:45:41.478860    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:46:03.389662 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:03.389629    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:46:13.175948 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:13.175911    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:46:22.527575 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.527540    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"]
Jul 20 22:46:22.530852 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.530836    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.534910 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.534886    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-ab4fc927-kserve-self-signed-certs\""
Jul 20 22:46:22.551473 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.551450    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"]
Jul 20 22:46:22.565701 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.565674    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-tmp-dir\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.565839 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.565712    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-home\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.565839 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.565764    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-dshm\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.565839 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.565780    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpzz4\" (UniqueName: \"kubernetes.io/projected/874eb3b8-ead3-45c4-a589-693729cde87a-kube-api-access-fpzz4\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.565839 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.565810    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-model-cache\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.565993 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.565850    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/874eb3b8-ead3-45c4-a589-693729cde87a-tls-certs\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.646302 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.646270    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:46:22.667062 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.667034    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-dshm\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.667226 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.667073    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpzz4\" (UniqueName: \"kubernetes.io/projected/874eb3b8-ead3-45c4-a589-693729cde87a-kube-api-access-fpzz4\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.667226 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.667198    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-model-cache\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.667331 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.667228    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/874eb3b8-ead3-45c4-a589-693729cde87a-tls-certs\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.667331 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.667324    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-tmp-dir\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.667420 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.667361    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-home\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.667543 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.667524    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-model-cache\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.667625 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.667610    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-home\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.667740 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.667695    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-tmp-dir\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.669361 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.669342    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-dshm\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.669630 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.669613    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/874eb3b8-ead3-45c4-a589-693729cde87a-tls-certs\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.676405 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.676379    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpzz4\" (UniqueName: \"kubernetes.io/projected/874eb3b8-ead3-45c4-a589-693729cde87a-kube-api-access-fpzz4\") pod \"e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") " pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.839310 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.839213    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:22.970272 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:22.970249    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"]
Jul 20 22:46:22.971759 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:46:22.971730    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod874eb3b8_ead3_45c4_a589_693729cde87a.slice/crio-23d858d18047901c36081b0b966bfd226433c5795c64abb02ba78135bf8d02f6 WatchSource:0}: Error finding container 23d858d18047901c36081b0b966bfd226433c5795c64abb02ba78135bf8d02f6: Status 404 returned error can't find the container with id 23d858d18047901c36081b0b966bfd226433c5795c64abb02ba78135bf8d02f6
Jul 20 22:46:23.857153 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:23.857117    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872" event={"ID":"874eb3b8-ead3-45c4-a589-693729cde87a","Type":"ContainerStarted","Data":"94043497091f1c426c120b43cf28944ea96ee97ca07bafaac1cfbbb6a91b2188"}
Jul 20 22:46:23.857153 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:23.857158    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872" event={"ID":"874eb3b8-ead3-45c4-a589-693729cde87a","Type":"ContainerStarted","Data":"23d858d18047901c36081b0b966bfd226433c5795c64abb02ba78135bf8d02f6"}
Jul 20 22:46:23.877343 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:23.877289    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872" podStartSLOduration=1.87727458 podStartE2EDuration="1.87727458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:46:23.875978033 +0000 UTC m=+1419.653682085" watchObservedRunningTime="2026-07-20 22:46:23.87727458 +0000 UTC m=+1419.654978622"
Jul 20 22:46:25.726013 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:25.725973    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:46:32.845546 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:32.845510    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:32.845546 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:32.845550    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:32.852409 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:32.852387    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:32.905732 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:32.905693    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:46:35.650494 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:35.650461    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:46:38.681794 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:38.681701    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:46:56.295427 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:56.295389    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:46:56.361942 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:46:56.361911    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:47:02.738438 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:02.738398    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"]
Jul 20 22:47:02.767506 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:02.767472    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:47:03.839415 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:03.839375    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:47:07.068770 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:07.068731    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:47:17.758709 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:17.758631    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872" podUID="874eb3b8-ead3-45c4-a589-693729cde87a" containerName="main" containerID="cri-o://94043497091f1c426c120b43cf28944ea96ee97ca07bafaac1cfbbb6a91b2188" gracePeriod=45
Jul 20 22:47:18.010922 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.010865    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:47:18.039387 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.039359    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-home\") pod \"874eb3b8-ead3-45c4-a589-693729cde87a\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") "
Jul 20 22:47:18.039536 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.039399    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-tmp-dir\") pod \"874eb3b8-ead3-45c4-a589-693729cde87a\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") "
Jul 20 22:47:18.039536 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.039421    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-model-cache\") pod \"874eb3b8-ead3-45c4-a589-693729cde87a\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") "
Jul 20 22:47:18.039536 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.039479    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpzz4\" (UniqueName: \"kubernetes.io/projected/874eb3b8-ead3-45c4-a589-693729cde87a-kube-api-access-fpzz4\") pod \"874eb3b8-ead3-45c4-a589-693729cde87a\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") "
Jul 20 22:47:18.039536 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.039527    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-dshm\") pod \"874eb3b8-ead3-45c4-a589-693729cde87a\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") "
Jul 20 22:47:18.039764 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.039565    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/874eb3b8-ead3-45c4-a589-693729cde87a-tls-certs\") pod \"874eb3b8-ead3-45c4-a589-693729cde87a\" (UID: \"874eb3b8-ead3-45c4-a589-693729cde87a\") "
Jul 20 22:47:18.039764 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.039645    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-home" (OuterVolumeSpecName: "home") pod "874eb3b8-ead3-45c4-a589-693729cde87a" (UID: "874eb3b8-ead3-45c4-a589-693729cde87a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 22:47:18.039764 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.039658    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "874eb3b8-ead3-45c4-a589-693729cde87a" (UID: "874eb3b8-ead3-45c4-a589-693729cde87a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 22:47:18.039764 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.039744    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-model-cache" (OuterVolumeSpecName: "model-cache") pod "874eb3b8-ead3-45c4-a589-693729cde87a" (UID: "874eb3b8-ead3-45c4-a589-693729cde87a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 22:47:18.039905 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.039862    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-home\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:47:18.039941 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.039903    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-tmp-dir\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:47:18.039941 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.039917    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-model-cache\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:47:18.041807 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.041782    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/874eb3b8-ead3-45c4-a589-693729cde87a-kube-api-access-fpzz4" (OuterVolumeSpecName: "kube-api-access-fpzz4") pod "874eb3b8-ead3-45c4-a589-693729cde87a" (UID: "874eb3b8-ead3-45c4-a589-693729cde87a"). InnerVolumeSpecName "kube-api-access-fpzz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 22:47:18.041899 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.041785    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-dshm" (OuterVolumeSpecName: "dshm") pod "874eb3b8-ead3-45c4-a589-693729cde87a" (UID: "874eb3b8-ead3-45c4-a589-693729cde87a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 22:47:18.041899 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.041807    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/874eb3b8-ead3-45c4-a589-693729cde87a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "874eb3b8-ead3-45c4-a589-693729cde87a" (UID: "874eb3b8-ead3-45c4-a589-693729cde87a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 22:47:18.093167 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.093128    2550 generic.go:356] "Generic (PLEG): container finished" podID="874eb3b8-ead3-45c4-a589-693729cde87a" containerID="94043497091f1c426c120b43cf28944ea96ee97ca07bafaac1cfbbb6a91b2188" exitCode=0
Jul 20 22:47:18.093352 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.093172    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"
Jul 20 22:47:18.093352 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.093219    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872" event={"ID":"874eb3b8-ead3-45c4-a589-693729cde87a","Type":"ContainerDied","Data":"94043497091f1c426c120b43cf28944ea96ee97ca07bafaac1cfbbb6a91b2188"}
Jul 20 22:47:18.093352 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.093267    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872" event={"ID":"874eb3b8-ead3-45c4-a589-693729cde87a","Type":"ContainerDied","Data":"23d858d18047901c36081b0b966bfd226433c5795c64abb02ba78135bf8d02f6"}
Jul 20 22:47:18.093352 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.093289    2550 scope.go:117] "RemoveContainer" containerID="94043497091f1c426c120b43cf28944ea96ee97ca07bafaac1cfbbb6a91b2188"
Jul 20 22:47:18.101751 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.101727    2550 scope.go:117] "RemoveContainer" containerID="94043497091f1c426c120b43cf28944ea96ee97ca07bafaac1cfbbb6a91b2188"
Jul 20 22:47:18.102023 ip-10-0-129-16 kubenswrapper[2550]: E0720 22:47:18.101996    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94043497091f1c426c120b43cf28944ea96ee97ca07bafaac1cfbbb6a91b2188\": container with ID starting with 94043497091f1c426c120b43cf28944ea96ee97ca07bafaac1cfbbb6a91b2188 not found: ID does not exist" containerID="94043497091f1c426c120b43cf28944ea96ee97ca07bafaac1cfbbb6a91b2188"
Jul 20 22:47:18.102065 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.102032    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94043497091f1c426c120b43cf28944ea96ee97ca07bafaac1cfbbb6a91b2188"} err="failed to get container status \"94043497091f1c426c120b43cf28944ea96ee97ca07bafaac1cfbbb6a91b2188\": rpc error: code = NotFound desc = could not find container \"94043497091f1c426c120b43cf28944ea96ee97ca07bafaac1cfbbb6a91b2188\": container with ID starting with 94043497091f1c426c120b43cf28944ea96ee97ca07bafaac1cfbbb6a91b2188 not found: ID does not exist"
Jul 20 22:47:18.114637 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.114609    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"]
Jul 20 22:47:18.116848 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.116824    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-ab4fc927-kserve-6b85c778c9-6c872"]
Jul 20 22:47:18.141106 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.141075    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpzz4\" (UniqueName: \"kubernetes.io/projected/874eb3b8-ead3-45c4-a589-693729cde87a-kube-api-access-fpzz4\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:47:18.141106 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.141103    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/874eb3b8-ead3-45c4-a589-693729cde87a-dshm\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:47:18.141257 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.141115    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/874eb3b8-ead3-45c4-a589-693729cde87a-tls-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:47:18.844517 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:18.844481    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="874eb3b8-ead3-45c4-a589-693729cde87a" path="/var/lib/kubelet/pods/874eb3b8-ead3-45c4-a589-693729cde87a/volumes"
Jul 20 22:47:21.974739 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:21.974682    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:47:31.076590 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:31.076549    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:47:43.683174 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:43.683129    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:47:44.862173 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:44.862142    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:47:44.863144 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:44.863122    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:47:58.742185 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:47:58.742146    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:48:08.076568 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:48:08.076470    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:48:22.088540 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:48:22.088493    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:48:31.577511 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:48:31.577476    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:48:42.304085 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:48:42.304042    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:48:54.374334 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:48:54.374299    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:49:09.584388 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:49:09.584351    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:49:17.787328 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:49:17.787297    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:49:30.662273 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:49:30.662237    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:49:46.873397 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:49:46.873320    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:49:59.576797 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:49:59.576755    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:50:09.170355 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:09.170317    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:50:12.387160 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:12.387123    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:19.042486 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:19.042449    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:23.170952 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:23.170906    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:50:27.305432 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:27.305400    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:30.860879 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:30.860845    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:32.470708 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:32.470676    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:50:32.548038 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:32.548002    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:35.475606 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:35.475566    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:35.574828 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:35.574792    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:35.902882 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:35.902782    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:42.004020 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:42.003985    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:42.066912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:42.066872    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:42.233285 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:42.233252    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:42.271024 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:42.270953    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:50:42.553635 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:42.553565    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:42.716230 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:42.716200    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:45.265490 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:45.265448    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:45.432257 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:45.432224    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:45.538480 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:45.538398    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:45.662290 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:45.662256    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:46.032787 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:46.032757    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:46.136853 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:46.136819    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:50:51.071596 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:50:51.071558    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:51:10.763242 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:10.763156    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr"]
Jul 20 22:51:10.763798 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:10.763634    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="874eb3b8-ead3-45c4-a589-693729cde87a" containerName="main"
Jul 20 22:51:10.763798 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:10.763654    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="874eb3b8-ead3-45c4-a589-693729cde87a" containerName="main"
Jul 20 22:51:10.763798 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:10.763782    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="874eb3b8-ead3-45c4-a589-693729cde87a" containerName="main"
Jul 20 22:51:10.766666 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:10.766644    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr"
Jul 20 22:51:10.769118 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:10.769098    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4f4tw\""
Jul 20 22:51:10.769226 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:10.769190    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 22:51:10.773703 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:10.773663    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr"]
Jul 20 22:51:10.836914 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:10.836881    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9ea77c4-1dd6-43f2-b77b-5902b1e113ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23458506-p5qjr\" (UID: \"b9ea77c4-1dd6-43f2-b77b-5902b1e113ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr"
Jul 20 22:51:10.937649 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:10.937618    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9ea77c4-1dd6-43f2-b77b-5902b1e113ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23458506-p5qjr\" (UID: \"b9ea77c4-1dd6-43f2-b77b-5902b1e113ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr"
Jul 20 22:51:10.938213 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:10.938194    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9ea77c4-1dd6-43f2-b77b-5902b1e113ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23458506-p5qjr\" (UID: \"b9ea77c4-1dd6-43f2-b77b-5902b1e113ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr"
Jul 20 22:51:11.077884 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:11.077795    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr"
Jul 20 22:51:11.203942 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:11.203917    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr"]
Jul 20 22:51:11.206370 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:51:11.206341    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9ea77c4_1dd6_43f2_b77b_5902b1e113ea.slice/crio-5f5c85c863ce689811a6d9b864c28ea8f7dd1b15d2a1591dda32b13a754f6028 WatchSource:0}: Error finding container 5f5c85c863ce689811a6d9b864c28ea8f7dd1b15d2a1591dda32b13a754f6028: Status 404 returned error can't find the container with id 5f5c85c863ce689811a6d9b864c28ea8f7dd1b15d2a1591dda32b13a754f6028
Jul 20 22:51:11.208313 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:11.208294    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 22:51:12.106739 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:12.106692    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr" event={"ID":"b9ea77c4-1dd6-43f2-b77b-5902b1e113ea","Type":"ContainerStarted","Data":"5f5c85c863ce689811a6d9b864c28ea8f7dd1b15d2a1591dda32b13a754f6028"}
Jul 20 22:51:13.854100 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:13.854071    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:13.955201 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:13.955169    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:14.119406 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:14.119327    2550 generic.go:356] "Generic (PLEG): container finished" podID="b9ea77c4-1dd6-43f2-b77b-5902b1e113ea" containerID="2533b8fa6e5ec196b7ec2770bf7032eaf4d6f1c97f68427182db2792bf6c1fbf" exitCode=0
Jul 20 22:51:14.119548 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:14.119419    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr" event={"ID":"b9ea77c4-1dd6-43f2-b77b-5902b1e113ea","Type":"ContainerDied","Data":"2533b8fa6e5ec196b7ec2770bf7032eaf4d6f1c97f68427182db2792bf6c1fbf"}
Jul 20 22:51:15.249275 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:15.249250    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr"
Jul 20 22:51:15.271684 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:15.271652    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9ea77c4-1dd6-43f2-b77b-5902b1e113ea-maas-api-service-ca\") pod \"b9ea77c4-1dd6-43f2-b77b-5902b1e113ea\" (UID: \"b9ea77c4-1dd6-43f2-b77b-5902b1e113ea\") "
Jul 20 22:51:15.272135 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:15.272090    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9ea77c4-1dd6-43f2-b77b-5902b1e113ea-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b9ea77c4-1dd6-43f2-b77b-5902b1e113ea" (UID: "b9ea77c4-1dd6-43f2-b77b-5902b1e113ea"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 22:51:15.372677 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:15.372642    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9ea77c4-1dd6-43f2-b77b-5902b1e113ea-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:51:16.129395 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:16.129369    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr"
Jul 20 22:51:16.129568 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:16.129395    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr" event={"ID":"b9ea77c4-1dd6-43f2-b77b-5902b1e113ea","Type":"ContainerDied","Data":"5f5c85c863ce689811a6d9b864c28ea8f7dd1b15d2a1591dda32b13a754f6028"}
Jul 20 22:51:16.129568 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:16.129442    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f5c85c863ce689811a6d9b864c28ea8f7dd1b15d2a1591dda32b13a754f6028"
Jul 20 22:51:21.209659 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:21.209606    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:24.070633 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:24.070598    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:51:25.850851 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:25.850819    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr"]
Jul 20 22:51:25.856329 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:25.856299    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23458506-p5qjr"]
Jul 20 22:51:25.857592 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:25.857567    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:26.009821 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:26.009784    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:26.245347 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:26.245311    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:26.499918 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:26.499825    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:26.601459 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:26.601414    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:26.711051 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:26.711014    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:26.844605 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:26.844525    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9ea77c4-1dd6-43f2-b77b-5902b1e113ea" path="/var/lib/kubelet/pods/b9ea77c4-1dd6-43f2-b77b-5902b1e113ea/volumes"
Jul 20 22:51:27.009546 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:27.009510    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:27.272585 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:27.272550    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:29.613973 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:29.613935    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:29.733821 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:29.733772    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:30.314030 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:30.313994    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:30.415375 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:30.415341    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:30.515833 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:30.515804    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:51:35.272207 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:35.272170    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:51:54.604793 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:54.604757    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm"]
Jul 20 22:51:54.605182 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:54.605088    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9ea77c4-1dd6-43f2-b77b-5902b1e113ea" containerName="revoke-keys"
Jul 20 22:51:54.605182 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:54.605100    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9ea77c4-1dd6-43f2-b77b-5902b1e113ea" containerName="revoke-keys"
Jul 20 22:51:54.605182 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:54.605163    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9ea77c4-1dd6-43f2-b77b-5902b1e113ea" containerName="revoke-keys"
Jul 20 22:51:54.607907 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:54.607889    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm"
Jul 20 22:51:54.610493 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:54.610470    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4f4tw\""
Jul 20 22:51:54.610614 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:54.610507    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 22:51:54.615555 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:54.615529    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm"]
Jul 20 22:51:54.714273 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:54.714236    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d22839-ce7f-4bbd-badc-c75ce79cab66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-55938262-fnnlm\" (UID: \"b6d22839-ce7f-4bbd-badc-c75ce79cab66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm"
Jul 20 22:51:54.815553 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:54.815506    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d22839-ce7f-4bbd-badc-c75ce79cab66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-55938262-fnnlm\" (UID: \"b6d22839-ce7f-4bbd-badc-c75ce79cab66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm"
Jul 20 22:51:54.816251 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:54.816223    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d22839-ce7f-4bbd-badc-c75ce79cab66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-55938262-fnnlm\" (UID: \"b6d22839-ce7f-4bbd-badc-c75ce79cab66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm"
Jul 20 22:51:54.919907 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:54.919807    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm"
Jul 20 22:51:55.046878 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:55.046852    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm"]
Jul 20 22:51:55.048650 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:51:55.048621    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d22839_ce7f_4bbd_badc_c75ce79cab66.slice/crio-7d7faa0d3292f589dcb14060b52eb11673679ec41007231c3fa197df6726cf49 WatchSource:0}: Error finding container 7d7faa0d3292f589dcb14060b52eb11673679ec41007231c3fa197df6726cf49: Status 404 returned error can't find the container with id 7d7faa0d3292f589dcb14060b52eb11673679ec41007231c3fa197df6726cf49
Jul 20 22:51:55.303167 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:55.303136    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm" event={"ID":"b6d22839-ce7f-4bbd-badc-c75ce79cab66","Type":"ContainerStarted","Data":"7d7faa0d3292f589dcb14060b52eb11673679ec41007231c3fa197df6726cf49"}
Jul 20 22:51:56.309106 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:56.309070    2550 generic.go:356] "Generic (PLEG): container finished" podID="b6d22839-ce7f-4bbd-badc-c75ce79cab66" containerID="ea20c9a370472a6f141a699d344e744e96b05a477e3319bb5ff4a341b27e590e" exitCode=0
Jul 20 22:51:56.309106 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:56.309112    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm" event={"ID":"b6d22839-ce7f-4bbd-badc-c75ce79cab66","Type":"ContainerDied","Data":"ea20c9a370472a6f141a699d344e744e96b05a477e3319bb5ff4a341b27e590e"}
Jul 20 22:51:57.449326 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:57.449298    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm"
Jul 20 22:51:57.640830 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:57.640756    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d22839-ce7f-4bbd-badc-c75ce79cab66-maas-api-service-ca\") pod \"b6d22839-ce7f-4bbd-badc-c75ce79cab66\" (UID: \"b6d22839-ce7f-4bbd-badc-c75ce79cab66\") "
Jul 20 22:51:57.641088 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:57.641066    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d22839-ce7f-4bbd-badc-c75ce79cab66-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b6d22839-ce7f-4bbd-badc-c75ce79cab66" (UID: "b6d22839-ce7f-4bbd-badc-c75ce79cab66"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 22:51:57.741362 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:57.741336    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d22839-ce7f-4bbd-badc-c75ce79cab66-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:51:58.318944 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:58.318915    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm"
Jul 20 22:51:58.319108 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:58.318915    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm" event={"ID":"b6d22839-ce7f-4bbd-badc-c75ce79cab66","Type":"ContainerDied","Data":"7d7faa0d3292f589dcb14060b52eb11673679ec41007231c3fa197df6726cf49"}
Jul 20 22:51:58.319108 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:51:58.319034    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d7faa0d3292f589dcb14060b52eb11673679ec41007231c3fa197df6726cf49"
Jul 20 22:52:05.066046 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:05.066007    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:08.064273 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:08.064237    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:52:09.687797 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:09.687765    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm"]
Jul 20 22:52:09.691091 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:09.691068    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:09.691225 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:09.691128    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-55938262-fnnlm"]
Jul 20 22:52:09.864773 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:09.864699    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:10.082948 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:10.082914    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:10.306967 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:10.306929    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:10.641775 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:10.641740    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:10.844267 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:10.844227    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6d22839-ce7f-4bbd-badc-c75ce79cab66" path="/var/lib/kubelet/pods/b6d22839-ce7f-4bbd-badc-c75ce79cab66/volumes"
Jul 20 22:52:10.858540 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:10.858514    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:12.773074 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:12.773033    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:12.862902 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:12.862865    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:13.002117 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:13.002070    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:13.157693 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:13.157614    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:13.588578 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:13.588542    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:13.689255 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:13.689224    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:19.473694 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:19.473659    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:52:38.560755 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:38.560651    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd"]
Jul 20 22:52:38.561189 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:38.561033    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6d22839-ce7f-4bbd-badc-c75ce79cab66" containerName="revoke-keys"
Jul 20 22:52:38.561189 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:38.561046    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6d22839-ce7f-4bbd-badc-c75ce79cab66" containerName="revoke-keys"
Jul 20 22:52:38.561189 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:38.561112    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6d22839-ce7f-4bbd-badc-c75ce79cab66" containerName="revoke-keys"
Jul 20 22:52:38.564095 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:38.564078    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd"
Jul 20 22:52:38.566468 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:38.566443    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 22:52:38.566600 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:38.566582    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4f4tw\""
Jul 20 22:52:38.571544 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:38.571521    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd"]
Jul 20 22:52:38.655278 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:38.655246    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba883ec-1e93-47e7-b2a7-10da403469d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cf9642a2-228dd\" (UID: \"6ba883ec-1e93-47e7-b2a7-10da403469d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd"
Jul 20 22:52:38.755828 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:38.755800    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba883ec-1e93-47e7-b2a7-10da403469d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cf9642a2-228dd\" (UID: \"6ba883ec-1e93-47e7-b2a7-10da403469d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd"
Jul 20 22:52:38.756385 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:38.756365    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba883ec-1e93-47e7-b2a7-10da403469d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cf9642a2-228dd\" (UID: \"6ba883ec-1e93-47e7-b2a7-10da403469d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd"
Jul 20 22:52:38.875033 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:38.874937    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd"
Jul 20 22:52:39.002222 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:39.002193    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd"]
Jul 20 22:52:39.004369 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:52:39.004345    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba883ec_1e93_47e7_b2a7_10da403469d6.slice/crio-308d36c8c746301eeb8fa06cb7617f13e0f7c853086cbb1331b0b221f9b80d0d WatchSource:0}: Error finding container 308d36c8c746301eeb8fa06cb7617f13e0f7c853086cbb1331b0b221f9b80d0d: Status 404 returned error can't find the container with id 308d36c8c746301eeb8fa06cb7617f13e0f7c853086cbb1331b0b221f9b80d0d
Jul 20 22:52:39.501570 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:39.501488    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd" event={"ID":"6ba883ec-1e93-47e7-b2a7-10da403469d6","Type":"ContainerStarted","Data":"e8f9143986d8c06362612af07e5733db00a9206f24767fcafe500a18550e9934"}
Jul 20 22:52:39.501570 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:39.501550    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd" event={"ID":"6ba883ec-1e93-47e7-b2a7-10da403469d6","Type":"ContainerStarted","Data":"308d36c8c746301eeb8fa06cb7617f13e0f7c853086cbb1331b0b221f9b80d0d"}
Jul 20 22:52:39.516825 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:39.516771    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd" podStartSLOduration=1.516757413 podStartE2EDuration="1.516757413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:52:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:52:39.515843277 +0000 UTC m=+1795.293547362" watchObservedRunningTime="2026-07-20 22:52:39.516757413 +0000 UTC m=+1795.294461465"
Jul 20 22:52:40.507280 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:40.507244    2550 generic.go:356] "Generic (PLEG): container finished" podID="6ba883ec-1e93-47e7-b2a7-10da403469d6" containerID="e8f9143986d8c06362612af07e5733db00a9206f24767fcafe500a18550e9934" exitCode=0
Jul 20 22:52:40.507676 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:40.507330    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd" event={"ID":"6ba883ec-1e93-47e7-b2a7-10da403469d6","Type":"ContainerDied","Data":"e8f9143986d8c06362612af07e5733db00a9206f24767fcafe500a18550e9934"}
Jul 20 22:52:41.647068 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:41.647044    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd"
Jul 20 22:52:41.780258 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:41.780162    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba883ec-1e93-47e7-b2a7-10da403469d6-maas-api-service-ca\") pod \"6ba883ec-1e93-47e7-b2a7-10da403469d6\" (UID: \"6ba883ec-1e93-47e7-b2a7-10da403469d6\") "
Jul 20 22:52:41.780512 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:41.780489    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ba883ec-1e93-47e7-b2a7-10da403469d6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ba883ec-1e93-47e7-b2a7-10da403469d6" (UID: "6ba883ec-1e93-47e7-b2a7-10da403469d6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 22:52:41.881534 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:41.881501    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba883ec-1e93-47e7-b2a7-10da403469d6-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:52:42.519592 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:42.519563    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd"
Jul 20 22:52:42.519592 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:42.519569    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd" event={"ID":"6ba883ec-1e93-47e7-b2a7-10da403469d6","Type":"ContainerDied","Data":"308d36c8c746301eeb8fa06cb7617f13e0f7c853086cbb1331b0b221f9b80d0d"}
Jul 20 22:52:42.519820 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:42.519618    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="308d36c8c746301eeb8fa06cb7617f13e0f7c853086cbb1331b0b221f9b80d0d"
Jul 20 22:52:44.889483 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:44.889446    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:52:44.891144 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:44.891120    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:52:49.026737 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:49.026686    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:51.668104 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:51.668070    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:52:53.638522 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:53.638485    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd"]
Jul 20 22:52:53.641796 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:53.641770    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cf9642a2-228dd"]
Jul 20 22:52:53.642677 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:53.642641    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:54.843424 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:54.843392    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:54.844737 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:54.844693    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ba883ec-1e93-47e7-b2a7-10da403469d6" path="/var/lib/kubelet/pods/6ba883ec-1e93-47e7-b2a7-10da403469d6/volumes"
Jul 20 22:52:55.089462 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:55.089426    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:55.298545 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:55.298516    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:55.627340 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:55.627265    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:56.093834 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:56.093799    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:58.077342 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:58.077305    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:58.177427 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:58.177396    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:58.284808 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:58.284772    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:58.620003 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:58.619968    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:59.111106 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:59.111074    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:52:59.212672 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:52:59.212641    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:01.064438 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:01.064402    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:53:10.301814 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:10.301778    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:10.400122 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:10.400081    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:20.848018 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:20.847982    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:20.947974 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:20.947929    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:22.910739 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:22.910684    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh"]
Jul 20 22:53:22.911124 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:22.911038    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ba883ec-1e93-47e7-b2a7-10da403469d6" containerName="revoke-keys"
Jul 20 22:53:22.911124 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:22.911049    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ba883ec-1e93-47e7-b2a7-10da403469d6" containerName="revoke-keys"
Jul 20 22:53:22.911124 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:22.911111    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ba883ec-1e93-47e7-b2a7-10da403469d6" containerName="revoke-keys"
Jul 20 22:53:22.914017 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:22.914000    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh"
Jul 20 22:53:22.916093 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:22.916071    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 22:53:22.916233 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:22.916143    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4f4tw\""
Jul 20 22:53:22.920235 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:22.920210    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh"]
Jul 20 22:53:23.030700 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:23.030670    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3d0061-2c3b-42be-89f9-9e611e3e19dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh\" (UID: \"0d3d0061-2c3b-42be-89f9-9e611e3e19dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh"
Jul 20 22:53:23.131685 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:23.131650    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3d0061-2c3b-42be-89f9-9e611e3e19dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh\" (UID: \"0d3d0061-2c3b-42be-89f9-9e611e3e19dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh"
Jul 20 22:53:23.132308 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:23.132287    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3d0061-2c3b-42be-89f9-9e611e3e19dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh\" (UID: \"0d3d0061-2c3b-42be-89f9-9e611e3e19dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh"
Jul 20 22:53:23.225116 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:23.225089    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh"
Jul 20 22:53:23.353262 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:23.353242    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh"]
Jul 20 22:53:23.356076 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:53:23.356044    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d3d0061_2c3b_42be_89f9_9e611e3e19dc.slice/crio-4c1b760437bc6068bdaf074ce3c9e72da562225868d69d70073a81dc7dc7b85e WatchSource:0}: Error finding container 4c1b760437bc6068bdaf074ce3c9e72da562225868d69d70073a81dc7dc7b85e: Status 404 returned error can't find the container with id 4c1b760437bc6068bdaf074ce3c9e72da562225868d69d70073a81dc7dc7b85e
Jul 20 22:53:23.698739 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:23.698684    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh" event={"ID":"0d3d0061-2c3b-42be-89f9-9e611e3e19dc","Type":"ContainerStarted","Data":"9640e649cc0c3b56c15d39115bacff17163a7722ccac014b342685c6053316c4"}
Jul 20 22:53:23.698904 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:23.698766    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh" event={"ID":"0d3d0061-2c3b-42be-89f9-9e611e3e19dc","Type":"ContainerStarted","Data":"4c1b760437bc6068bdaf074ce3c9e72da562225868d69d70073a81dc7dc7b85e"}
Jul 20 22:53:23.713700 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:23.713652    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh" podStartSLOduration=1.713636878 podStartE2EDuration="1.713636878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:53:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:53:23.711935407 +0000 UTC m=+1839.489639450" watchObservedRunningTime="2026-07-20 22:53:23.713636878 +0000 UTC m=+1839.491340931"
Jul 20 22:53:24.705262 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:24.705232    2550 generic.go:356] "Generic (PLEG): container finished" podID="0d3d0061-2c3b-42be-89f9-9e611e3e19dc" containerID="9640e649cc0c3b56c15d39115bacff17163a7722ccac014b342685c6053316c4" exitCode=0
Jul 20 22:53:24.705629 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:24.705285    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh" event={"ID":"0d3d0061-2c3b-42be-89f9-9e611e3e19dc","Type":"ContainerDied","Data":"9640e649cc0c3b56c15d39115bacff17163a7722ccac014b342685c6053316c4"}
Jul 20 22:53:25.837051 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:25.837028    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh"
Jul 20 22:53:25.955527 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:25.955495    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3d0061-2c3b-42be-89f9-9e611e3e19dc-maas-api-service-ca\") pod \"0d3d0061-2c3b-42be-89f9-9e611e3e19dc\" (UID: \"0d3d0061-2c3b-42be-89f9-9e611e3e19dc\") "
Jul 20 22:53:25.955891 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:25.955869    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d3d0061-2c3b-42be-89f9-9e611e3e19dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0d3d0061-2c3b-42be-89f9-9e611e3e19dc" (UID: "0d3d0061-2c3b-42be-89f9-9e611e3e19dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 22:53:26.056932 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:26.056866    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3d0061-2c3b-42be-89f9-9e611e3e19dc-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:53:26.714692 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:26.714652    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh" event={"ID":"0d3d0061-2c3b-42be-89f9-9e611e3e19dc","Type":"ContainerDied","Data":"4c1b760437bc6068bdaf074ce3c9e72da562225868d69d70073a81dc7dc7b85e"}
Jul 20 22:53:26.714692 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:26.714679    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh"
Jul 20 22:53:26.714911 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:26.714710    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c1b760437bc6068bdaf074ce3c9e72da562225868d69d70073a81dc7dc7b85e"
Jul 20 22:53:33.368570 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:33.368506    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:36.066262 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:36.066224    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:53:38.018938 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:38.018902    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:38.019367 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:38.019173    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh"]
Jul 20 22:53:38.021948 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:38.021919    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0332ce3d-f45jh"]
Jul 20 22:53:38.178897 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:38.178858    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:38.445856 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:38.445766    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:38.844155 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:38.844126    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d3d0061-2c3b-42be-89f9-9e611e3e19dc" path="/var/lib/kubelet/pods/0d3d0061-2c3b-42be-89f9-9e611e3e19dc/volumes"
Jul 20 22:53:38.915706 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:38.915670    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:39.010819 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:39.010786    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:39.114967 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:39.114890    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:40.645927 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:40.645884    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:47.738470 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:47.738436    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:47.849000 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:47.848969    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:47.970089 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:47.970057    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:48.063547 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:48.063470    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:53:53.558600 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:53:53.558567    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:54:19.545128 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:54:19.545093    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:54:26.204447 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:54:26.204410    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:54:26.307209 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:54:26.307170    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:54:26.409339 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:54:26.409294    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:54:42.283571 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:54:42.283529    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:54:48.323508 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:54:48.323469    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:54:48.390912 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:54:48.390867    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:54:48.555034 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:54:48.554995    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:54:48.696421 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:54:48.696349    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:54:59.087470 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:54:59.087439    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:54:59.901286 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:54:59.901254    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:06.261156 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:06.261125    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:06.375688 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:06.375650    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:06.462298 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:06.462267    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:06.562168 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:06.562081    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:07.319186 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:07.319147    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:07.438370 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:07.438329    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:07.538996 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:07.538963    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:09.140364 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:09.140325    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:09.567823 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:09.567789    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:55:15.096205 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:15.096171    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:16.167841 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:16.167807    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:20.788398 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:20.788347    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:55:23.031211 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:23.031171    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:23.126308 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:23.126272    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:23.258891 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:23.258855    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:23.600354 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:23.600324    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:23.761988 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:23.761939    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:26.487569 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:26.487530    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:26.583581 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:26.583548    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:26.709936 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:26.709901    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:27.143970 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:27.143930    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:27.262552 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:27.262518    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:27.362686 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:27.362655    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:31.061204 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:31.061173    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:31.606561 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:31.606521    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:31.707891 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:31.707859    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:32.122362 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:32.122328    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:33.671352 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:33.671318    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:55:37.994701 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:37.994561    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:38.063410 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.063365    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:38.332677 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.332599    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9"]
Jul 20 22:55:38.332957 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.332944    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d3d0061-2c3b-42be-89f9-9e611e3e19dc" containerName="revoke-keys"
Jul 20 22:55:38.333004 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.332958    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d3d0061-2c3b-42be-89f9-9e611e3e19dc" containerName="revoke-keys"
Jul 20 22:55:38.333042 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.333024    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d3d0061-2c3b-42be-89f9-9e611e3e19dc" containerName="revoke-keys"
Jul 20 22:55:38.335560 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.335543    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9"
Jul 20 22:55:38.337835 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.337819    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 22:55:38.337936 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.337857    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4f4tw\""
Jul 20 22:55:38.342974 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.342955    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9"]
Jul 20 22:55:38.461963 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.461931    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b173056f-a78e-4519-b5a9-e9f76f072db4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9\" (UID: \"b173056f-a78e-4519-b5a9-e9f76f072db4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9"
Jul 20 22:55:38.562894 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.562862    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b173056f-a78e-4519-b5a9-e9f76f072db4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9\" (UID: \"b173056f-a78e-4519-b5a9-e9f76f072db4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9"
Jul 20 22:55:38.563444 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.563426    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b173056f-a78e-4519-b5a9-e9f76f072db4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9\" (UID: \"b173056f-a78e-4519-b5a9-e9f76f072db4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9"
Jul 20 22:55:38.647001 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.646923    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9"
Jul 20 22:55:38.780470 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:38.780441    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9"]
Jul 20 22:55:38.782794 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:55:38.782766    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb173056f_a78e_4519_b5a9_e9f76f072db4.slice/crio-f46acf58b87209a8d7be1b69e3f55c9161f1a00982ec38b4a917dab06237de0e WatchSource:0}: Error finding container f46acf58b87209a8d7be1b69e3f55c9161f1a00982ec38b4a917dab06237de0e: Status 404 returned error can't find the container with id f46acf58b87209a8d7be1b69e3f55c9161f1a00982ec38b4a917dab06237de0e
Jul 20 22:55:39.305629 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:39.305550    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9" event={"ID":"b173056f-a78e-4519-b5a9-e9f76f072db4","Type":"ContainerStarted","Data":"8a94953fbb214707b1b01018ad5111bb0eedd565e819004bf22361950d021130"}
Jul 20 22:55:39.305629 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:39.305607    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9" event={"ID":"b173056f-a78e-4519-b5a9-e9f76f072db4","Type":"ContainerStarted","Data":"f46acf58b87209a8d7be1b69e3f55c9161f1a00982ec38b4a917dab06237de0e"}
Jul 20 22:55:39.321410 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:39.321360    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9" podStartSLOduration=1.321344458 podStartE2EDuration="1.321344458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:55:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:55:39.32025887 +0000 UTC m=+1975.097962932" watchObservedRunningTime="2026-07-20 22:55:39.321344458 +0000 UTC m=+1975.099048510"
Jul 20 22:55:40.316057 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:40.316018    2550 generic.go:356] "Generic (PLEG): container finished" podID="b173056f-a78e-4519-b5a9-e9f76f072db4" containerID="8a94953fbb214707b1b01018ad5111bb0eedd565e819004bf22361950d021130" exitCode=0
Jul 20 22:55:40.316057 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:40.316061    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9" event={"ID":"b173056f-a78e-4519-b5a9-e9f76f072db4","Type":"ContainerDied","Data":"8a94953fbb214707b1b01018ad5111bb0eedd565e819004bf22361950d021130"}
Jul 20 22:55:41.447551 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:41.447528    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9"
Jul 20 22:55:41.586696 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:41.586614    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b173056f-a78e-4519-b5a9-e9f76f072db4-maas-api-service-ca\") pod \"b173056f-a78e-4519-b5a9-e9f76f072db4\" (UID: \"b173056f-a78e-4519-b5a9-e9f76f072db4\") "
Jul 20 22:55:41.586972 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:41.586946    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b173056f-a78e-4519-b5a9-e9f76f072db4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b173056f-a78e-4519-b5a9-e9f76f072db4" (UID: "b173056f-a78e-4519-b5a9-e9f76f072db4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 22:55:41.687899 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:41.687864    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b173056f-a78e-4519-b5a9-e9f76f072db4-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:55:42.325818 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:42.325787    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9"
Jul 20 22:55:42.325818 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:42.325804    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9" event={"ID":"b173056f-a78e-4519-b5a9-e9f76f072db4","Type":"ContainerDied","Data":"f46acf58b87209a8d7be1b69e3f55c9161f1a00982ec38b4a917dab06237de0e"}
Jul 20 22:55:42.326017 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:42.325857    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f46acf58b87209a8d7be1b69e3f55c9161f1a00982ec38b4a917dab06237de0e"
Jul 20 22:55:48.885704 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:48.885674    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:50.362577 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:50.362542    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:50.420133 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:50.420087    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:51.760882 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:51.760845    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:55:53.414528 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:53.414487    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9"]
Jul 20 22:55:53.416864 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:53.416832    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-36f27dea-qdnf9"]
Jul 20 22:55:53.417962 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:53.417927    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:53.791667 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:53.791631    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:54.845471 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:54.845436    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b173056f-a78e-4519-b5a9-e9f76f072db4" path="/var/lib/kubelet/pods/b173056f-a78e-4519-b5a9-e9f76f072db4/volumes"
Jul 20 22:55:59.814479 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:59.814443    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:55:59.874590 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:55:59.874555    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:00.103276 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:00.103200    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:00.325023 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:00.324991    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:00.465317 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:00.465287    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:00.564850 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:00.564821    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:00.682999 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:00.682969    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:06.400978 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:06.400940    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:06.510017 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:06.509983    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:06.601435 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:06.601400    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:06.757613 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:06.757583    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:12.644204 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:12.644170    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:12.705942 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:12.705906    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:19.429773 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:19.429707    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:19.745985 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:19.745953    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:19.929436 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:19.929400    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:22.703427 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:22.703391    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:22.795800 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:22.795768    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:22.860616 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:22.860586    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:23.022068 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:23.021987    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:23.410272 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:23.410187    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:23.511270 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:23.511236    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:27.180630 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:27.180590    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:27.462427 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:27.462389    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:56:40.287937 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.287900    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"]
Jul 20 22:56:40.288408 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.288394    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b173056f-a78e-4519-b5a9-e9f76f072db4" containerName="revoke-keys"
Jul 20 22:56:40.288480 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.288415    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b173056f-a78e-4519-b5a9-e9f76f072db4" containerName="revoke-keys"
Jul 20 22:56:40.288537 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.288517    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b173056f-a78e-4519-b5a9-e9f76f072db4" containerName="revoke-keys"
Jul 20 22:56:40.291595 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.291572    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.293812 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.293792    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5418bc43\"/\"openshift-service-ca.crt\""
Jul 20 22:56:40.294764 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.294742    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5418bc43\"/\"default-dockercfg-j9v5b\""
Jul 20 22:56:40.294860 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.294774    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5418bc43\"/\"e2e-lifecycle-model-5418bc43-kserve-self-signed-certs\""
Jul 20 22:56:40.294860 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.294789    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5418bc43\"/\"kube-root-ca.crt\""
Jul 20 22:56:40.301337 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.301301    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"]
Jul 20 22:56:40.319990 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.319962    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:56:40.455163 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.455129    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-home\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.455341 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.455187    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6qbn\" (UniqueName: \"kubernetes.io/projected/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-kube-api-access-n6qbn\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.455341 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.455223    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-dshm\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.455341 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.455242    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-tls-certs\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.455341 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.455263    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-tmp-dir\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.455341 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.455289    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-model-cache\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.556117 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.556028    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-model-cache\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.556117 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.556087    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-home\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.556349 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.556157    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6qbn\" (UniqueName: \"kubernetes.io/projected/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-kube-api-access-n6qbn\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.556349 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.556190    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-dshm\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.556349 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.556213    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-tls-certs\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.556349 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.556244    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-tmp-dir\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.556561 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.556510    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-home\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.556620 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.556560    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-model-cache\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.556669 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.556620    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-tmp-dir\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.558462 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.558437    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-dshm\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.558613 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.558593    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-tls-certs\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.563486 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.563463    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6qbn\" (UniqueName: \"kubernetes.io/projected/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-kube-api-access-n6qbn\") pod \"e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") " pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.602994 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.602961    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:40.732319 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:40.732292    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"]
Jul 20 22:56:40.734437 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:56:40.734409    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3a16d25_cb3b_43ca_bd4c_f7d589e6fdcb.slice/crio-ea8d341d311d7ae18a0a5720ba3eed149d18568c9defd856dd4eaaa8096a5498 WatchSource:0}: Error finding container ea8d341d311d7ae18a0a5720ba3eed149d18568c9defd856dd4eaaa8096a5498: Status 404 returned error can't find the container with id ea8d341d311d7ae18a0a5720ba3eed149d18568c9defd856dd4eaaa8096a5498
Jul 20 22:56:41.600784 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:41.600739    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z" event={"ID":"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb","Type":"ContainerStarted","Data":"2ec8fdc6311a630a92d4e32e6fc37e3b5e57af36a20759ae6867ba8425dee93f"}
Jul 20 22:56:41.600784 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:41.600790    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z" event={"ID":"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb","Type":"ContainerStarted","Data":"ea8d341d311d7ae18a0a5720ba3eed149d18568c9defd856dd4eaaa8096a5498"}
Jul 20 22:56:41.619391 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:41.619341    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z" podStartSLOduration=1.619327693 podStartE2EDuration="1.619327693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:56:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:56:41.617383647 +0000 UTC m=+2037.395087712" watchObservedRunningTime="2026-07-20 22:56:41.619327693 +0000 UTC m=+2037.397031744"
Jul 20 22:56:42.365343 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:42.365306    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:56:50.604020 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:50.603978    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:50.604512 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:50.604047    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:50.616427 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:50.616399    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:50.652651 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:50.652623    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:56:57.261042 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:56:57.261001    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:57:00.418054 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:00.418009    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz"]
Jul 20 22:57:00.421420 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:00.421395    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz"
Jul 20 22:57:00.423761 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:00.423735    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 22:57:00.423880 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:00.423819    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4f4tw\""
Jul 20 22:57:00.427951 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:00.427926    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz"]
Jul 20 22:57:00.527771 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:00.527710    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a48cd96-af0a-4119-9fbb-ec13b2e6b19c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5418bc43-plqxz\" (UID: \"3a48cd96-af0a-4119-9fbb-ec13b2e6b19c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz"
Jul 20 22:57:00.628856 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:00.628825    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a48cd96-af0a-4119-9fbb-ec13b2e6b19c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5418bc43-plqxz\" (UID: \"3a48cd96-af0a-4119-9fbb-ec13b2e6b19c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz"
Jul 20 22:57:00.629473 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:00.629452    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a48cd96-af0a-4119-9fbb-ec13b2e6b19c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5418bc43-plqxz\" (UID: \"3a48cd96-af0a-4119-9fbb-ec13b2e6b19c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz"
Jul 20 22:57:00.732778 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:00.732741    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz"
Jul 20 22:57:00.860626 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:00.860515    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz"]
Jul 20 22:57:00.863275 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:57:00.863241    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a48cd96_af0a_4119_9fbb_ec13b2e6b19c.slice/crio-a2b29658ea883b37420d73f65c0c9ae55d31c80e0f045fca5398782b7fcee949 WatchSource:0}: Error finding container a2b29658ea883b37420d73f65c0c9ae55d31c80e0f045fca5398782b7fcee949: Status 404 returned error can't find the container with id a2b29658ea883b37420d73f65c0c9ae55d31c80e0f045fca5398782b7fcee949
Jul 20 22:57:01.692987 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:01.692951    2550 generic.go:356] "Generic (PLEG): container finished" podID="3a48cd96-af0a-4119-9fbb-ec13b2e6b19c" containerID="15cdbeb67dfb8ff03d9f72ef9afb0a1fb684b5b074d5184b2d0d47f26b92c217" exitCode=0
Jul 20 22:57:01.693347 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:01.693003    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz" event={"ID":"3a48cd96-af0a-4119-9fbb-ec13b2e6b19c","Type":"ContainerDied","Data":"15cdbeb67dfb8ff03d9f72ef9afb0a1fb684b5b074d5184b2d0d47f26b92c217"}
Jul 20 22:57:01.693347 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:01.693045    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz" event={"ID":"3a48cd96-af0a-4119-9fbb-ec13b2e6b19c","Type":"ContainerStarted","Data":"a2b29658ea883b37420d73f65c0c9ae55d31c80e0f045fca5398782b7fcee949"}
Jul 20 22:57:02.828188 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:02.828164    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz"
Jul 20 22:57:02.948800 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:02.948705    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a48cd96-af0a-4119-9fbb-ec13b2e6b19c-maas-api-service-ca\") pod \"3a48cd96-af0a-4119-9fbb-ec13b2e6b19c\" (UID: \"3a48cd96-af0a-4119-9fbb-ec13b2e6b19c\") "
Jul 20 22:57:02.948968 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:02.948948    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a48cd96-af0a-4119-9fbb-ec13b2e6b19c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3a48cd96-af0a-4119-9fbb-ec13b2e6b19c" (UID: "3a48cd96-af0a-4119-9fbb-ec13b2e6b19c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 22:57:02.949182 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:02.949147    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a48cd96-af0a-4119-9fbb-ec13b2e6b19c-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:57:03.706009 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:03.705975    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz"
Jul 20 22:57:03.706203 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:03.706015    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz" event={"ID":"3a48cd96-af0a-4119-9fbb-ec13b2e6b19c","Type":"ContainerDied","Data":"a2b29658ea883b37420d73f65c0c9ae55d31c80e0f045fca5398782b7fcee949"}
Jul 20 22:57:03.706203 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:03.706091    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2b29658ea883b37420d73f65c0c9ae55d31c80e0f045fca5398782b7fcee949"
Jul 20 22:57:14.477464 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:14.477428    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:57:15.907422 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:15.907390    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:57:20.497082 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:20.497036    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz"]
Jul 20 22:57:20.498846 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:20.498818    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5418bc43-plqxz"]
Jul 20 22:57:20.502356 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:20.502331    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:57:20.844930 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:20.844856    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a48cd96-af0a-4119-9fbb-ec13b2e6b19c" path="/var/lib/kubelet/pods/3a48cd96-af0a-4119-9fbb-ec13b2e6b19c/volumes"
Jul 20 22:57:22.723009 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:22.722967    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:57:23.567075 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:23.567037    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:57:27.705880 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:27.705841    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"]
Jul 20 22:57:27.801007 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:27.800972    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:57:42.726415 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.726372    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z" podUID="e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb" containerName="main" containerID="cri-o://2ec8fdc6311a630a92d4e32e6fc37e3b5e57af36a20759ae6867ba8425dee93f" gracePeriod=45
Jul 20 22:57:42.885304 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.885274    2550 generic.go:356] "Generic (PLEG): container finished" podID="e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb" containerID="2ec8fdc6311a630a92d4e32e6fc37e3b5e57af36a20759ae6867ba8425dee93f" exitCode=0
Jul 20 22:57:42.885475 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.885357    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z" event={"ID":"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb","Type":"ContainerDied","Data":"2ec8fdc6311a630a92d4e32e6fc37e3b5e57af36a20759ae6867ba8425dee93f"}
Jul 20 22:57:42.970291 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.970268    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:57:42.980596 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.980552    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-home\") pod \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") "
Jul 20 22:57:42.980596 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.980579    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-dshm\") pod \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") "
Jul 20 22:57:42.980596 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.980597    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-tls-certs\") pod \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") "
Jul 20 22:57:42.980780 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.980630    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-tmp-dir\") pod \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") "
Jul 20 22:57:42.980780 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.980663    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6qbn\" (UniqueName: \"kubernetes.io/projected/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-kube-api-access-n6qbn\") pod \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") "
Jul 20 22:57:42.980780 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.980700    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-model-cache\") pod \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\" (UID: \"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb\") "
Jul 20 22:57:42.980918 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.980880    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-home" (OuterVolumeSpecName: "home") pod "e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb" (UID: "e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 22:57:42.980999 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.980967    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb" (UID: "e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 22:57:42.980999 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.980984    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-home\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:57:42.981126 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.981088    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-model-cache" (OuterVolumeSpecName: "model-cache") pod "e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb" (UID: "e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 22:57:42.982739 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.982687    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb" (UID: "e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 22:57:42.982824 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.982784    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-kube-api-access-n6qbn" (OuterVolumeSpecName: "kube-api-access-n6qbn") pod "e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb" (UID: "e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb"). InnerVolumeSpecName "kube-api-access-n6qbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 22:57:42.982942 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:42.982928    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-dshm" (OuterVolumeSpecName: "dshm") pod "e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb" (UID: "e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 22:57:43.082078 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:43.082048    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-dshm\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:57:43.082078 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:43.082071    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-tls-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:57:43.082078 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:43.082081    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-tmp-dir\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:57:43.082308 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:43.082089    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6qbn\" (UniqueName: \"kubernetes.io/projected/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-kube-api-access-n6qbn\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:57:43.082308 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:43.082097    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb-model-cache\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 22:57:43.890948 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:43.890919    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"
Jul 20 22:57:43.891362 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:43.890918    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z" event={"ID":"e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb","Type":"ContainerDied","Data":"ea8d341d311d7ae18a0a5720ba3eed149d18568c9defd856dd4eaaa8096a5498"}
Jul 20 22:57:43.891362 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:43.891038    2550 scope.go:117] "RemoveContainer" containerID="2ec8fdc6311a630a92d4e32e6fc37e3b5e57af36a20759ae6867ba8425dee93f"
Jul 20 22:57:43.914368 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:43.914343    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"]
Jul 20 22:57:43.918410 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:43.918387    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-5418bc43/e2e-lifecycle-model-5418bc43-kserve-75b94b5d4c-hqd6z"]
Jul 20 22:57:44.845148 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:44.845114    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb" path="/var/lib/kubelet/pods/e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb/volumes"
Jul 20 22:57:44.927945 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:44.927920    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:57:44.929790 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:44.929766    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 22:57:44.960532 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:57:44.960508    2550 scope.go:117] "RemoveContainer" containerID="2533b8fa6e5ec196b7ec2770bf7032eaf4d6f1c97f68427182db2792bf6c1fbf"
Jul 20 22:58:03.856240 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:03.856201    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:05.017134 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:05.017104    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:11.217369 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:11.217337    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:11.320273 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:11.320240    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:11.512680 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:11.512601    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:11.738705 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:11.738665    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:12.766316 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:12.766269    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:12.864947 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:12.864913    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:12.961905 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:12.961871    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:13.674083 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:13.674040    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:13.768020 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:13.767987    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:13.869676 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:13.869643    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:15.224404 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:15.224356    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:15.330933 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:15.330897    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:58:21.468358 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:21.468324    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:21.540393 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:21.540359    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:22.147463 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:22.147427    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:27.772920 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:27.772872    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:58:28.261466 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:28.261429    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:28.366051 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:28.366014    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:28.461596 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:28.461561    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:38.958657 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:38.958582    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:39.281170 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:39.281085    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:58:44.981068 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:44.981042    2550 scope.go:117] "RemoveContainer" containerID="e8f9143986d8c06362612af07e5733db00a9206f24767fcafe500a18550e9934"
Jul 20 22:58:44.988871 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:58:44.988849    2550 scope.go:117] "RemoveContainer" containerID="ea20c9a370472a6f141a699d344e744e96b05a477e3319bb5ff4a341b27e590e"
Jul 20 22:59:00.222525 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:00.222452    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:02.960639 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:02.960599    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:06.778058 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:06.778019    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:06.878323 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:06.878287    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:07.120405 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:07.120310    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:07.404416 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:07.404331    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:07.504119 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:07.504089    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:10.693064 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:10.693032    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:10.775462 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:10.775431    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:10.843524 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:10.843488    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:11.278326 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:11.278207    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:11.378734 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:11.378688    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:12.314422 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:12.314387    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:15.172090 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:15.172048    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:15.505535 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:15.505495    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:15.799011 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:15.798934    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:15.972839 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:15.972797    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:59:15.989407 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:15.989375    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:18.759488 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:18.759443    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:18.849042 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:18.849007    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:19.027197 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:19.027092    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:19.602443 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:19.602241    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:19.701605 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:19.701549    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:23.215006 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:23.214973    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:24.768846 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:24.768808    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:59:32.215766 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:32.215734    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:32.325788 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:32.325745    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:33.310014 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:33.309977    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:36.401635 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.401597    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"]
Jul 20 22:59:36.402020 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.401986    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb" containerName="main"
Jul 20 22:59:36.402020 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.401999    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb" containerName="main"
Jul 20 22:59:36.402020 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.402021    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a48cd96-af0a-4119-9fbb-ec13b2e6b19c" containerName="revoke-keys"
Jul 20 22:59:36.402127 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.402026    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a48cd96-af0a-4119-9fbb-ec13b2e6b19c" containerName="revoke-keys"
Jul 20 22:59:36.402127 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.402079    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3a16d25-cb3b-43ca-bd4c-f7d589e6fdcb" containerName="main"
Jul 20 22:59:36.402127 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.402088    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a48cd96-af0a-4119-9fbb-ec13b2e6b19c" containerName="revoke-keys"
Jul 20 22:59:36.405156 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.405132    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.408171 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.408136    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-222578\"/\"kube-root-ca.crt\""
Jul 20 22:59:36.408311 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.408204    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-222578\"/\"openshift-service-ca.crt\""
Jul 20 22:59:36.408311 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.408263    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-222578\"/\"test-model-222578-kserve-self-signed-certs\""
Jul 20 22:59:36.408311 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.408295    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-222578\"/\"default-dockercfg-bd57x\""
Jul 20 22:59:36.417091 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.417058    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"]
Jul 20 22:59:36.425348 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.425321    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:36.519754 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.519695    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-dshm\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.519932 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.519788    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-tmp-dir\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.519932 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.519832    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2d9r\" (UniqueName: \"kubernetes.io/projected/33f39021-933e-4f63-a22a-fa09adb1119c-kube-api-access-j2d9r\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.519932 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.519854    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f39021-933e-4f63-a22a-fa09adb1119c-tls-certs\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.520062 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.519942    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-home\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.520062 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.519978    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-model-cache\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.621075 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.621042    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-home\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.621075 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.621082    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-model-cache\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.621331 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.621122    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-dshm\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.621331 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.621141    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-tmp-dir\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.621331 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.621161    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2d9r\" (UniqueName: \"kubernetes.io/projected/33f39021-933e-4f63-a22a-fa09adb1119c-kube-api-access-j2d9r\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.621331 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.621184    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f39021-933e-4f63-a22a-fa09adb1119c-tls-certs\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.621787 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.621709    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-model-cache\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.621920 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.621836    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-tmp-dir\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.621920 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.621843    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-home\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.623976 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.623956    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-dshm\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.624243 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.624225    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f39021-933e-4f63-a22a-fa09adb1119c-tls-certs\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.629249 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.629224    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2d9r\" (UniqueName: \"kubernetes.io/projected/33f39021-933e-4f63-a22a-fa09adb1119c-kube-api-access-j2d9r\") pod \"test-model-222578-kserve-54cfbdd95c-44xh2\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") " pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.714380 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.714348    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:36.850410 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:36.850385    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"]
Jul 20 22:59:36.853195 ip-10-0-129-16 kubenswrapper[2550]: W0720 22:59:36.853165    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33f39021_933e_4f63_a22a_fa09adb1119c.slice/crio-5ab5c460a02547e0840e31c080bfefb1c7d5dca3fc157f29db0d40a0cabbd9ba WatchSource:0}: Error finding container 5ab5c460a02547e0840e31c080bfefb1c7d5dca3fc157f29db0d40a0cabbd9ba: Status 404 returned error can't find the container with id 5ab5c460a02547e0840e31c080bfefb1c7d5dca3fc157f29db0d40a0cabbd9ba
Jul 20 22:59:37.415636 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:37.415596    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2" event={"ID":"33f39021-933e-4f63-a22a-fa09adb1119c","Type":"ContainerStarted","Data":"3b1b7594a3eb25b611800b6cd2751c92fd45dee57de935e2fa5a7654358b0ddb"}
Jul 20 22:59:37.415636 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:37.415644    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2" event={"ID":"33f39021-933e-4f63-a22a-fa09adb1119c","Type":"ContainerStarted","Data":"5ab5c460a02547e0840e31c080bfefb1c7d5dca3fc157f29db0d40a0cabbd9ba"}
Jul 20 22:59:37.453657 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:37.453603    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2" podStartSLOduration=1.453586856 podStartE2EDuration="1.453586856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:59:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:59:37.452031158 +0000 UTC m=+2213.229735214" watchObservedRunningTime="2026-07-20 22:59:37.453586856 +0000 UTC m=+2213.231290907"
Jul 20 22:59:40.167975 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:40.167932    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 22:59:40.368897 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:40.368859    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:40.469342 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:40.469312    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:44.286384 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:44.286332    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:44.373330 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:44.373294    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 22:59:45.011130 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:45.011091    2550 scope.go:117] "RemoveContainer" containerID="9640e649cc0c3b56c15d39115bacff17163a7722ccac014b342685c6053316c4"
Jul 20 22:59:46.714797 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:46.714754    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:46.714797 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:46.714803    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:46.727206 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:46.727171    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:47.471473 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:47.471440    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 22:59:50.767829 ip-10-0-129-16 kubenswrapper[2550]: I0720 22:59:50.767789    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:00:00.146786 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.146746    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-222578-29743140-4ss7p"]
Jul 20 23:00:00.156665 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.156633    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-35607f-29743140-m748w"]
Jul 20 23:00:00.156826 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.156755    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-222578-29743140-4ss7p"
Jul 20 23:00:00.159271 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.159246    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4f4tw\""
Jul 20 23:00:00.160257 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.160234    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743140-jb4ws"]
Jul 20 23:00:00.160612 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.160423    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-35607f-29743140-m748w"
Jul 20 23:00:00.164024 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.163426    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-222578-29743140-4ss7p"]
Jul 20 23:00:00.164024 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.163523    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743140-jb4ws"
Jul 20 23:00:00.164024 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.163641    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-35607f-29743140-m748w"]
Jul 20 23:00:00.165702 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.165679    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743140-jb4ws"]
Jul 20 23:00:00.173885 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.173864    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-222578-29743140-4ss7p"
Jul 20 23:00:00.180373 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.180343    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-35607f-29743140-m748w"
Jul 20 23:00:00.207384 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.207362    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743140-jb4ws"
Jul 20 23:00:00.326747 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.326697    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-222578-29743140-4ss7p"]
Jul 20 23:00:00.329888 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:00:00.329859    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4582140_ef1e_463d_96f2_a195e48f5ae9.slice/crio-52d1ea6ab28fb8581f4ac285c6797c68901798e0ae9ca9522253f392bb6d3906 WatchSource:0}: Error finding container 52d1ea6ab28fb8581f4ac285c6797c68901798e0ae9ca9522253f392bb6d3906: Status 404 returned error can't find the container with id 52d1ea6ab28fb8581f4ac285c6797c68901798e0ae9ca9522253f392bb6d3906
Jul 20 23:00:00.389498 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.389474    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743140-jb4ws"]
Jul 20 23:00:00.391153 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:00:00.391128    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29edf151_bdba_45e6_a271_6763690c79be.slice/crio-b7b63d7278e3605560f93544bb3c844a5a4cea32ca7422a81a6b038f3cf2507f WatchSource:0}: Error finding container b7b63d7278e3605560f93544bb3c844a5a4cea32ca7422a81a6b038f3cf2507f: Status 404 returned error can't find the container with id b7b63d7278e3605560f93544bb3c844a5a4cea32ca7422a81a6b038f3cf2507f
Jul 20 23:00:00.520739 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.520677    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743140-jb4ws" event={"ID":"29edf151-bdba-45e6-a271-6763690c79be","Type":"ContainerStarted","Data":"b7b63d7278e3605560f93544bb3c844a5a4cea32ca7422a81a6b038f3cf2507f"}
Jul 20 23:00:00.521801 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.521778    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-222578-29743140-4ss7p" event={"ID":"f4582140-ef1e-463d-96f2-a195e48f5ae9","Type":"ContainerStarted","Data":"52d1ea6ab28fb8581f4ac285c6797c68901798e0ae9ca9522253f392bb6d3906"}
Jul 20 23:00:00.561919 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:00.561899    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-35607f-29743140-m748w"]
Jul 20 23:00:00.564023 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:00:00.564000    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec527ebc_0442_42f3_a7fa_34566711103d.slice/crio-f8c03aa2090e84c32a07a3ea7f230e0900167b570eb7609e211eb1e428c35f33 WatchSource:0}: Error finding container f8c03aa2090e84c32a07a3ea7f230e0900167b570eb7609e211eb1e428c35f33: Status 404 returned error can't find the container with id f8c03aa2090e84c32a07a3ea7f230e0900167b570eb7609e211eb1e428c35f33
Jul 20 23:00:01.528074 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:01.528032    2550 generic.go:356] "Generic (PLEG): container finished" podID="f4582140-ef1e-463d-96f2-a195e48f5ae9" containerID="bcc38cfd965bdf1bf167249aaf44a696bb7060a230fa41c7dd267900eac69102" exitCode=0
Jul 20 23:00:01.528484 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:01.528121    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-222578-29743140-4ss7p" event={"ID":"f4582140-ef1e-463d-96f2-a195e48f5ae9","Type":"ContainerDied","Data":"bcc38cfd965bdf1bf167249aaf44a696bb7060a230fa41c7dd267900eac69102"}
Jul 20 23:00:01.529569 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:01.529548    2550 generic.go:356] "Generic (PLEG): container finished" podID="ec527ebc-0442-42f3-a7fa-34566711103d" containerID="aecdb32f423203f281300b0e04cf7fde31219c0f846ae05fc3ad5deb748aaad3" exitCode=0
Jul 20 23:00:01.529649 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:01.529629    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-35607f-29743140-m748w" event={"ID":"ec527ebc-0442-42f3-a7fa-34566711103d","Type":"ContainerDied","Data":"aecdb32f423203f281300b0e04cf7fde31219c0f846ae05fc3ad5deb748aaad3"}
Jul 20 23:00:01.529689 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:01.529674    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-35607f-29743140-m748w" event={"ID":"ec527ebc-0442-42f3-a7fa-34566711103d","Type":"ContainerStarted","Data":"f8c03aa2090e84c32a07a3ea7f230e0900167b570eb7609e211eb1e428c35f33"}
Jul 20 23:00:01.531084 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:01.531062    2550 generic.go:356] "Generic (PLEG): container finished" podID="29edf151-bdba-45e6-a271-6763690c79be" containerID="66e8ecd3befaf4432a737a9917e063807e0785ed7eccc73a66d167db4ea4390f" exitCode=0
Jul 20 23:00:01.531199 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:01.531121    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743140-jb4ws" event={"ID":"29edf151-bdba-45e6-a271-6763690c79be","Type":"ContainerDied","Data":"66e8ecd3befaf4432a737a9917e063807e0785ed7eccc73a66d167db4ea4390f"}
Jul 20 23:00:02.719780 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:02.719756    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743140-jb4ws"
Jul 20 23:00:02.722368 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:02.722339    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-35607f-29743140-m748w"
Jul 20 23:00:02.727948 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:02.727930    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-222578-29743140-4ss7p"
Jul 20 23:00:03.542842 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:03.542809    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743140-jb4ws"
Jul 20 23:00:03.543146 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:03.542847    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743140-jb4ws" event={"ID":"29edf151-bdba-45e6-a271-6763690c79be","Type":"ContainerDied","Data":"b7b63d7278e3605560f93544bb3c844a5a4cea32ca7422a81a6b038f3cf2507f"}
Jul 20 23:00:03.543146 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:03.542901    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7b63d7278e3605560f93544bb3c844a5a4cea32ca7422a81a6b038f3cf2507f"
Jul 20 23:00:03.544197 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:03.544177    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-222578-29743140-4ss7p"
Jul 20 23:00:03.544197 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:03.544182    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-222578-29743140-4ss7p" event={"ID":"f4582140-ef1e-463d-96f2-a195e48f5ae9","Type":"ContainerDied","Data":"52d1ea6ab28fb8581f4ac285c6797c68901798e0ae9ca9522253f392bb6d3906"}
Jul 20 23:00:03.544369 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:03.544215    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52d1ea6ab28fb8581f4ac285c6797c68901798e0ae9ca9522253f392bb6d3906"
Jul 20 23:00:03.545493 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:03.545471    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-35607f-29743140-m748w" event={"ID":"ec527ebc-0442-42f3-a7fa-34566711103d","Type":"ContainerDied","Data":"f8c03aa2090e84c32a07a3ea7f230e0900167b570eb7609e211eb1e428c35f33"}
Jul 20 23:00:03.545493 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:03.545484    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-35607f-29743140-m748w"
Jul 20 23:00:03.545623 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:03.545506    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f8c03aa2090e84c32a07a3ea7f230e0900167b570eb7609e211eb1e428c35f33"
Jul 20 23:00:06.789500 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:06.789421    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:00:10.768787 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:10.768748    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:00:24.965122 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:24.965083    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:00:47.718377 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:47.718345    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"]
Jul 20 23:00:47.744160 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:47.744128    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:00:48.809138 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.809100    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz"]
Jul 20 23:00:48.809592 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.809571    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec527ebc-0442-42f3-a7fa-34566711103d" containerName="cleanup"
Jul 20 23:00:48.809664 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.809595    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec527ebc-0442-42f3-a7fa-34566711103d" containerName="cleanup"
Jul 20 23:00:48.809664 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.809617    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4582140-ef1e-463d-96f2-a195e48f5ae9" containerName="cleanup"
Jul 20 23:00:48.809664 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.809626    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4582140-ef1e-463d-96f2-a195e48f5ae9" containerName="cleanup"
Jul 20 23:00:48.809664 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.809654    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29edf151-bdba-45e6-a271-6763690c79be" containerName="cleanup"
Jul 20 23:00:48.809664 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.809663    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="29edf151-bdba-45e6-a271-6763690c79be" containerName="cleanup"
Jul 20 23:00:48.809946 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.809762    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4582140-ef1e-463d-96f2-a195e48f5ae9" containerName="cleanup"
Jul 20 23:00:48.809946 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.809778    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec527ebc-0442-42f3-a7fa-34566711103d" containerName="cleanup"
Jul 20 23:00:48.809946 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.809792    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="29edf151-bdba-45e6-a271-6763690c79be" containerName="cleanup"
Jul 20 23:00:48.813862 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.813842    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz"
Jul 20 23:00:48.820341 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.820319    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4f4tw\""
Jul 20 23:00:48.820430 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.820328    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 23:00:48.825888 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.825854    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz"]
Jul 20 23:00:48.940544 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:48.940508    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c5558-1381-4d0e-8233-623a24f2be2c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-222578-q7dmz\" (UID: \"bf0c5558-1381-4d0e-8233-623a24f2be2c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz"
Jul 20 23:00:49.041387 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:49.041352    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c5558-1381-4d0e-8233-623a24f2be2c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-222578-q7dmz\" (UID: \"bf0c5558-1381-4d0e-8233-623a24f2be2c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz"
Jul 20 23:00:49.042122 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:49.042100    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c5558-1381-4d0e-8233-623a24f2be2c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-222578-q7dmz\" (UID: \"bf0c5558-1381-4d0e-8233-623a24f2be2c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz"
Jul 20 23:00:49.123929 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:49.123867    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz"
Jul 20 23:00:49.252543 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:49.252518    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz"]
Jul 20 23:00:49.255297 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:00:49.255264    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0c5558_1381_4d0e_8233_623a24f2be2c.slice/crio-578c6d702677f6c3018cf1ef0435dff3dda5f49e473b1a343048f6bfe10cab1c WatchSource:0}: Error finding container 578c6d702677f6c3018cf1ef0435dff3dda5f49e473b1a343048f6bfe10cab1c: Status 404 returned error can't find the container with id 578c6d702677f6c3018cf1ef0435dff3dda5f49e473b1a343048f6bfe10cab1c
Jul 20 23:00:49.572237 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:49.572209    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:00:49.757872 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:49.757803    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz" event={"ID":"bf0c5558-1381-4d0e-8233-623a24f2be2c","Type":"ContainerStarted","Data":"34ed1fad1b69e8d2e1e04e9b2227167baff70f0abf60495d7c9bcb58ce6a13ee"}
Jul 20 23:00:49.757872 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:49.757854    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz" event={"ID":"bf0c5558-1381-4d0e-8233-623a24f2be2c","Type":"ContainerStarted","Data":"578c6d702677f6c3018cf1ef0435dff3dda5f49e473b1a343048f6bfe10cab1c"}
Jul 20 23:00:49.771677 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:49.771636    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz" podStartSLOduration=1.771622708 podStartE2EDuration="1.771622708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 23:00:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 23:00:49.770355067 +0000 UTC m=+2285.548059119" watchObservedRunningTime="2026-07-20 23:00:49.771622708 +0000 UTC m=+2285.549326751"
Jul 20 23:00:50.763931 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:50.763897    2550 generic.go:356] "Generic (PLEG): container finished" podID="bf0c5558-1381-4d0e-8233-623a24f2be2c" containerID="34ed1fad1b69e8d2e1e04e9b2227167baff70f0abf60495d7c9bcb58ce6a13ee" exitCode=0
Jul 20 23:00:50.764307 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:50.763979    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz" event={"ID":"bf0c5558-1381-4d0e-8233-623a24f2be2c","Type":"ContainerDied","Data":"34ed1fad1b69e8d2e1e04e9b2227167baff70f0abf60495d7c9bcb58ce6a13ee"}
Jul 20 23:00:51.899216 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:51.899191    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz"
Jul 20 23:00:51.966835 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:51.966800    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c5558-1381-4d0e-8233-623a24f2be2c-maas-api-service-ca\") pod \"bf0c5558-1381-4d0e-8233-623a24f2be2c\" (UID: \"bf0c5558-1381-4d0e-8233-623a24f2be2c\") "
Jul 20 23:00:51.967157 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:51.967135    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf0c5558-1381-4d0e-8233-623a24f2be2c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bf0c5558-1381-4d0e-8233-623a24f2be2c" (UID: "bf0c5558-1381-4d0e-8233-623a24f2be2c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 23:00:52.067374 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:52.067307    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c5558-1381-4d0e-8233-623a24f2be2c-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:00:52.460343 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:52.460306    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:00:52.774920 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:52.774840    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz" event={"ID":"bf0c5558-1381-4d0e-8233-623a24f2be2c","Type":"ContainerDied","Data":"578c6d702677f6c3018cf1ef0435dff3dda5f49e473b1a343048f6bfe10cab1c"}
Jul 20 23:00:52.774920 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:52.774888    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="578c6d702677f6c3018cf1ef0435dff3dda5f49e473b1a343048f6bfe10cab1c"
Jul 20 23:00:52.774920 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:52.774847    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz"
Jul 20 23:00:59.271978 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:59.271945    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:00:59.275694 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:59.275666    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-222578-29743140-4ss7p"]
Jul 20 23:00:59.278235 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:00:59.278213    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-222578-29743140-4ss7p"]
Jul 20 23:01:00.844290 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:00.844258    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4582140-ef1e-463d-96f2-a195e48f5ae9" path="/var/lib/kubelet/pods/f4582140-ef1e-463d-96f2-a195e48f5ae9/volumes"
Jul 20 23:01:02.738871 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:02.738822    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2" podUID="33f39021-933e-4f63-a22a-fa09adb1119c" containerName="main" containerID="cri-o://3b1b7594a3eb25b611800b6cd2751c92fd45dee57de935e2fa5a7654358b0ddb" gracePeriod=45
Jul 20 23:01:02.992939 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:02.992880    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 23:01:03.052380 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.052348    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-dshm\") pod \"33f39021-933e-4f63-a22a-fa09adb1119c\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") "
Jul 20 23:01:03.052541 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.052415    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f39021-933e-4f63-a22a-fa09adb1119c-tls-certs\") pod \"33f39021-933e-4f63-a22a-fa09adb1119c\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") "
Jul 20 23:01:03.052541 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.052453    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-model-cache\") pod \"33f39021-933e-4f63-a22a-fa09adb1119c\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") "
Jul 20 23:01:03.052541 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.052507    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-tmp-dir\") pod \"33f39021-933e-4f63-a22a-fa09adb1119c\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") "
Jul 20 23:01:03.052755 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.052682    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2d9r\" (UniqueName: \"kubernetes.io/projected/33f39021-933e-4f63-a22a-fa09adb1119c-kube-api-access-j2d9r\") pod \"33f39021-933e-4f63-a22a-fa09adb1119c\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") "
Jul 20 23:01:03.052821 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.052772    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "33f39021-933e-4f63-a22a-fa09adb1119c" (UID: "33f39021-933e-4f63-a22a-fa09adb1119c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:01:03.052821 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.052778    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-home\") pod \"33f39021-933e-4f63-a22a-fa09adb1119c\" (UID: \"33f39021-933e-4f63-a22a-fa09adb1119c\") "
Jul 20 23:01:03.052917 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.052839    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-model-cache" (OuterVolumeSpecName: "model-cache") pod "33f39021-933e-4f63-a22a-fa09adb1119c" (UID: "33f39021-933e-4f63-a22a-fa09adb1119c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:01:03.053009 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.052988    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-home" (OuterVolumeSpecName: "home") pod "33f39021-933e-4f63-a22a-fa09adb1119c" (UID: "33f39021-933e-4f63-a22a-fa09adb1119c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:01:03.053130 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.053114    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-model-cache\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:01:03.053130 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.053134    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-tmp-dir\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:01:03.053238 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.053146    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-home\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:01:03.054725 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.054688    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33f39021-933e-4f63-a22a-fa09adb1119c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "33f39021-933e-4f63-a22a-fa09adb1119c" (UID: "33f39021-933e-4f63-a22a-fa09adb1119c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 23:01:03.054952 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.054934    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-dshm" (OuterVolumeSpecName: "dshm") pod "33f39021-933e-4f63-a22a-fa09adb1119c" (UID: "33f39021-933e-4f63-a22a-fa09adb1119c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:01:03.055000 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.054946    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33f39021-933e-4f63-a22a-fa09adb1119c-kube-api-access-j2d9r" (OuterVolumeSpecName: "kube-api-access-j2d9r") pod "33f39021-933e-4f63-a22a-fa09adb1119c" (UID: "33f39021-933e-4f63-a22a-fa09adb1119c"). InnerVolumeSpecName "kube-api-access-j2d9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 23:01:03.153959 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.153929    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f39021-933e-4f63-a22a-fa09adb1119c-dshm\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:01:03.153959 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.153954    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f39021-933e-4f63-a22a-fa09adb1119c-tls-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:01:03.153959 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.153965    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2d9r\" (UniqueName: \"kubernetes.io/projected/33f39021-933e-4f63-a22a-fa09adb1119c-kube-api-access-j2d9r\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:01:03.827973 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.827928    2550 generic.go:356] "Generic (PLEG): container finished" podID="33f39021-933e-4f63-a22a-fa09adb1119c" containerID="3b1b7594a3eb25b611800b6cd2751c92fd45dee57de935e2fa5a7654358b0ddb" exitCode=0
Jul 20 23:01:03.828403 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.828005    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"
Jul 20 23:01:03.828403 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.828002    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2" event={"ID":"33f39021-933e-4f63-a22a-fa09adb1119c","Type":"ContainerDied","Data":"3b1b7594a3eb25b611800b6cd2751c92fd45dee57de935e2fa5a7654358b0ddb"}
Jul 20 23:01:03.828403 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.828134    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2" event={"ID":"33f39021-933e-4f63-a22a-fa09adb1119c","Type":"ContainerDied","Data":"5ab5c460a02547e0840e31c080bfefb1c7d5dca3fc157f29db0d40a0cabbd9ba"}
Jul 20 23:01:03.828403 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.828157    2550 scope.go:117] "RemoveContainer" containerID="3b1b7594a3eb25b611800b6cd2751c92fd45dee57de935e2fa5a7654358b0ddb"
Jul 20 23:01:03.837003 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.836981    2550 scope.go:117] "RemoveContainer" containerID="3b1b7594a3eb25b611800b6cd2751c92fd45dee57de935e2fa5a7654358b0ddb"
Jul 20 23:01:03.837281 ip-10-0-129-16 kubenswrapper[2550]: E0720 23:01:03.837261    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b1b7594a3eb25b611800b6cd2751c92fd45dee57de935e2fa5a7654358b0ddb\": container with ID starting with 3b1b7594a3eb25b611800b6cd2751c92fd45dee57de935e2fa5a7654358b0ddb not found: ID does not exist" containerID="3b1b7594a3eb25b611800b6cd2751c92fd45dee57de935e2fa5a7654358b0ddb"
Jul 20 23:01:03.837343 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.837292    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b1b7594a3eb25b611800b6cd2751c92fd45dee57de935e2fa5a7654358b0ddb"} err="failed to get container status \"3b1b7594a3eb25b611800b6cd2751c92fd45dee57de935e2fa5a7654358b0ddb\": rpc error: code = NotFound desc = could not find container \"3b1b7594a3eb25b611800b6cd2751c92fd45dee57de935e2fa5a7654358b0ddb\": container with ID starting with 3b1b7594a3eb25b611800b6cd2751c92fd45dee57de935e2fa5a7654358b0ddb not found: ID does not exist"
Jul 20 23:01:03.847848 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.847820    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"]
Jul 20 23:01:03.851590 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.851567    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-222578/test-model-222578-kserve-54cfbdd95c-44xh2"]
Jul 20 23:01:03.860622 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.860601    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz"]
Jul 20 23:01:03.862613 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:03.862592    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-222578-q7dmz"]
Jul 20 23:01:04.266588 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:04.266556    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:04.328037 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:04.328008    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:04.844892 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:04.844858    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33f39021-933e-4f63-a22a-fa09adb1119c" path="/var/lib/kubelet/pods/33f39021-933e-4f63-a22a-fa09adb1119c/volumes"
Jul 20 23:01:04.845255 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:04.845167    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf0c5558-1381-4d0e-8233-623a24f2be2c" path="/var/lib/kubelet/pods/bf0c5558-1381-4d0e-8233-623a24f2be2c/volumes"
Jul 20 23:01:08.764197 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:08.764160    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:01:10.577249 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:10.577206    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:10.675700 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:10.675659    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:10.820929 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:10.820879    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:11.063534 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.063495    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:11.170109 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.170071    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg"]
Jul 20 23:01:11.170455 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.170438    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33f39021-933e-4f63-a22a-fa09adb1119c" containerName="main"
Jul 20 23:01:11.170528 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.170457    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="33f39021-933e-4f63-a22a-fa09adb1119c" containerName="main"
Jul 20 23:01:11.170528 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.170494    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf0c5558-1381-4d0e-8233-623a24f2be2c" containerName="revoke-keys"
Jul 20 23:01:11.170528 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.170505    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf0c5558-1381-4d0e-8233-623a24f2be2c" containerName="revoke-keys"
Jul 20 23:01:11.170680 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.170592    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="33f39021-933e-4f63-a22a-fa09adb1119c" containerName="main"
Jul 20 23:01:11.170680 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.170606    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf0c5558-1381-4d0e-8233-623a24f2be2c" containerName="revoke-keys"
Jul 20 23:01:11.174830 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.174810    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg"
Jul 20 23:01:11.176988 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.176966    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 23:01:11.177067 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.176977    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4f4tw\""
Jul 20 23:01:11.180493 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.180467    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg"]
Jul 20 23:01:11.319555 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.319480    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e9a5f5-f47c-4241-90e3-7da6678de25c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg\" (UID: \"28e9a5f5-f47c-4241-90e3-7da6678de25c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg"
Jul 20 23:01:11.420248 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.420221    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e9a5f5-f47c-4241-90e3-7da6678de25c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg\" (UID: \"28e9a5f5-f47c-4241-90e3-7da6678de25c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg"
Jul 20 23:01:11.420828 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.420807    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e9a5f5-f47c-4241-90e3-7da6678de25c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg\" (UID: \"28e9a5f5-f47c-4241-90e3-7da6678de25c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg"
Jul 20 23:01:11.485412 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.485388    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg"
Jul 20 23:01:11.614912 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.614887    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg"]
Jul 20 23:01:11.617330 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:01:11.617301    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28e9a5f5_f47c_4241_90e3_7da6678de25c.slice/crio-745b6b37437f5302e7be3a10689845d063e12c325b34310e842d7ae0aedc6cba WatchSource:0}: Error finding container 745b6b37437f5302e7be3a10689845d063e12c325b34310e842d7ae0aedc6cba: Status 404 returned error can't find the container with id 745b6b37437f5302e7be3a10689845d063e12c325b34310e842d7ae0aedc6cba
Jul 20 23:01:11.693251 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.693225    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:11.795661 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.795629    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:11.866633 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:11.866570    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg" event={"ID":"28e9a5f5-f47c-4241-90e3-7da6678de25c","Type":"ContainerStarted","Data":"745b6b37437f5302e7be3a10689845d063e12c325b34310e842d7ae0aedc6cba"}
Jul 20 23:01:12.749822 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:12.749763    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:12.871869 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:12.871839    2550 generic.go:356] "Generic (PLEG): container finished" podID="28e9a5f5-f47c-4241-90e3-7da6678de25c" containerID="1efff9403f3449b9500b7d9b586caa30a9bd1b2b9bffff03f396a6585328417e" exitCode=0
Jul 20 23:01:12.872032 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:12.871929    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg" event={"ID":"28e9a5f5-f47c-4241-90e3-7da6678de25c","Type":"ContainerDied","Data":"1efff9403f3449b9500b7d9b586caa30a9bd1b2b9bffff03f396a6585328417e"}
Jul 20 23:01:14.004466 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:14.004443    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg"
Jul 20 23:01:14.140476 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:14.140397    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e9a5f5-f47c-4241-90e3-7da6678de25c-maas-api-service-ca\") pod \"28e9a5f5-f47c-4241-90e3-7da6678de25c\" (UID: \"28e9a5f5-f47c-4241-90e3-7da6678de25c\") "
Jul 20 23:01:14.140745 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:14.140700    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28e9a5f5-f47c-4241-90e3-7da6678de25c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "28e9a5f5-f47c-4241-90e3-7da6678de25c" (UID: "28e9a5f5-f47c-4241-90e3-7da6678de25c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 23:01:14.240999 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:14.240972    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e9a5f5-f47c-4241-90e3-7da6678de25c-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:01:14.887888 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:14.887862    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg"
Jul 20 23:01:14.888005 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:14.887856    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg" event={"ID":"28e9a5f5-f47c-4241-90e3-7da6678de25c","Type":"ContainerDied","Data":"745b6b37437f5302e7be3a10689845d063e12c325b34310e842d7ae0aedc6cba"}
Jul 20 23:01:14.888058 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:14.888007    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="745b6b37437f5302e7be3a10689845d063e12c325b34310e842d7ae0aedc6cba"
Jul 20 23:01:21.714356 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:21.714322    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-35607f-29743140-m748w"]
Jul 20 23:01:21.714925 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:21.714835    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:21.716457 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:21.716435    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-35607f-29743140-m748w"]
Jul 20 23:01:22.844166 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:22.844135    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec527ebc-0442-42f3-a7fa-34566711103d" path="/var/lib/kubelet/pods/ec527ebc-0442-42f3-a7fa-34566711103d/volumes"
Jul 20 23:01:24.768262 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:24.768220    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:01:26.244899 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:26.244868    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg"]
Jul 20 23:01:26.245328 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:26.245048    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:26.247186 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:26.247161    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-35607f-8lzgg"]
Jul 20 23:01:26.619428 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:26.619348    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:26.844135 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:26.844106    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28e9a5f5-f47c-4241-90e3-7da6678de25c" path="/var/lib/kubelet/pods/28e9a5f5-f47c-4241-90e3-7da6678de25c/volumes"
Jul 20 23:01:32.481190 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:32.481148    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:32.629349 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:32.629314    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:32.746443 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:32.746360    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:32.975768 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:32.975736    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:33.297139 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:33.297088    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:33.615851 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:33.615772    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:34.089174 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:34.089128    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:36.531163 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:36.531113    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:36.610120 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:36.610090    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:36.680329 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:36.680252    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:36.963811 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:36.963779    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:37.292087 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:37.292001    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:37.392331 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:37.392301    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:41.012384 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:41.012352    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:41.474646 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:41.474615    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:41.784218 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:41.784125    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:42.008823 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:42.008786    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:43.664293 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:43.664258    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:01:44.602691 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:44.602653    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:44.782905 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:44.782873    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:44.880352 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:44.880269    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:45.035115 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:45.035087    2550 scope.go:117] "RemoveContainer" containerID="8a94953fbb214707b1b01018ad5111bb0eedd565e819004bf22361950d021130"
Jul 20 23:01:45.277699 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:45.277666    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:45.379675 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:45.379645    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:45.699453 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:45.699425    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:49.067046 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:49.067012    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:57.024053 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.024015    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9"]
Jul 20 23:01:57.024497 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.024352    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e9a5f5-f47c-4241-90e3-7da6678de25c" containerName="revoke-keys"
Jul 20 23:01:57.024497 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.024365    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e9a5f5-f47c-4241-90e3-7da6678de25c" containerName="revoke-keys"
Jul 20 23:01:57.024497 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.024438    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="28e9a5f5-f47c-4241-90e3-7da6678de25c" containerName="revoke-keys"
Jul 20 23:01:57.027460 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.027438    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9"
Jul 20 23:01:57.031923 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.031897    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4f4tw\""
Jul 20 23:01:57.032081 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.031912    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 23:01:57.039081 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.039056    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9"]
Jul 20 23:01:57.077973 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.077953    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba69c259-9a60-4df3-97c2-47b405cbfd37-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-238953-gprc9\" (UID: \"ba69c259-9a60-4df3-97c2-47b405cbfd37\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9"
Jul 20 23:01:57.178511 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.178486    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba69c259-9a60-4df3-97c2-47b405cbfd37-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-238953-gprc9\" (UID: \"ba69c259-9a60-4df3-97c2-47b405cbfd37\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9"
Jul 20 23:01:57.179108 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.179090    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba69c259-9a60-4df3-97c2-47b405cbfd37-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-238953-gprc9\" (UID: \"ba69c259-9a60-4df3-97c2-47b405cbfd37\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9"
Jul 20 23:01:57.337822 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.337749    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9"
Jul 20 23:01:57.476916 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.476892    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9"]
Jul 20 23:01:57.478915 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:01:57.478884    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba69c259_9a60_4df3_97c2_47b405cbfd37.slice/crio-d9da088ad0a17f1704f8ad87be10d5bc51407140b43090aa25f783d5365853b1 WatchSource:0}: Error finding container d9da088ad0a17f1704f8ad87be10d5bc51407140b43090aa25f783d5365853b1: Status 404 returned error can't find the container with id d9da088ad0a17f1704f8ad87be10d5bc51407140b43090aa25f783d5365853b1
Jul 20 23:01:57.674254 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.674224    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:57.773812 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:57.773789    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:01:58.087691 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:58.087659    2550 generic.go:356] "Generic (PLEG): container finished" podID="ba69c259-9a60-4df3-97c2-47b405cbfd37" containerID="f7d5f26e7235e922d87a2b0680303f73713974dd91b544d0c94e9a4618c9e9c8" exitCode=0
Jul 20 23:01:58.088033 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:58.087696    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9" event={"ID":"ba69c259-9a60-4df3-97c2-47b405cbfd37","Type":"ContainerDied","Data":"f7d5f26e7235e922d87a2b0680303f73713974dd91b544d0c94e9a4618c9e9c8"}
Jul 20 23:01:58.088033 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:58.087765    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9" event={"ID":"ba69c259-9a60-4df3-97c2-47b405cbfd37","Type":"ContainerStarted","Data":"d9da088ad0a17f1704f8ad87be10d5bc51407140b43090aa25f783d5365853b1"}
Jul 20 23:01:59.218667 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:59.218643    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9"
Jul 20 23:01:59.296165 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:59.296141    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba69c259-9a60-4df3-97c2-47b405cbfd37-maas-api-service-ca\") pod \"ba69c259-9a60-4df3-97c2-47b405cbfd37\" (UID: \"ba69c259-9a60-4df3-97c2-47b405cbfd37\") "
Jul 20 23:01:59.296453 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:59.296430    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba69c259-9a60-4df3-97c2-47b405cbfd37-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ba69c259-9a60-4df3-97c2-47b405cbfd37" (UID: "ba69c259-9a60-4df3-97c2-47b405cbfd37"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 23:01:59.396969 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:59.396910    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba69c259-9a60-4df3-97c2-47b405cbfd37-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:01:59.469233 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:01:59.469204    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:02:00.098931 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:00.098903    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9"
Jul 20 23:02:00.099103 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:00.098928    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9" event={"ID":"ba69c259-9a60-4df3-97c2-47b405cbfd37","Type":"ContainerDied","Data":"d9da088ad0a17f1704f8ad87be10d5bc51407140b43090aa25f783d5365853b1"}
Jul 20 23:02:00.099103 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:00.098972    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9da088ad0a17f1704f8ad87be10d5bc51407140b43090aa25f783d5365853b1"
Jul 20 23:02:06.065771 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:06.065738    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:06.165587 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:06.165559    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:07.711242 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:07.711211    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:07.812861 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:07.812834    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:12.983053 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:12.983020    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:16.057469 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:16.057434    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:02:17.107861 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:17.107823    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:17.117712 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:17.117677    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9"]
Jul 20 23:02:17.123254 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:17.123224    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-238953-gprc9"]
Jul 20 23:02:17.492931 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:17.492892    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:18.843981 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:18.843948    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba69c259-9a60-4df3-97c2-47b405cbfd37" path="/var/lib/kubelet/pods/ba69c259-9a60-4df3-97c2-47b405cbfd37/volumes"
Jul 20 23:02:23.294040 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:23.294003    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:23.361223 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:23.361188    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:23.559694 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:23.559616    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:23.780296 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:23.780264    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:23.903880 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:23.903802    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg"]
Jul 20 23:02:23.904172 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:23.904153    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba69c259-9a60-4df3-97c2-47b405cbfd37" containerName="revoke-keys"
Jul 20 23:02:23.904172 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:23.904168    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba69c259-9a60-4df3-97c2-47b405cbfd37" containerName="revoke-keys"
Jul 20 23:02:23.904254 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:23.904229    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba69c259-9a60-4df3-97c2-47b405cbfd37" containerName="revoke-keys"
Jul 20 23:02:23.907187 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:23.907172    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg"
Jul 20 23:02:23.909419 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:23.909398    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 23:02:23.909519 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:23.909494    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4f4tw\""
Jul 20 23:02:23.914338 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:23.914246    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg"]
Jul 20 23:02:23.999759 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:23.999708    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be3e3ec-d455-4999-94cd-da635509fe8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg\" (UID: \"0be3e3ec-d455-4999-94cd-da635509fe8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg"
Jul 20 23:02:24.100824 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:24.100794    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be3e3ec-d455-4999-94cd-da635509fe8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg\" (UID: \"0be3e3ec-d455-4999-94cd-da635509fe8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg"
Jul 20 23:02:24.101396 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:24.101375    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be3e3ec-d455-4999-94cd-da635509fe8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg\" (UID: \"0be3e3ec-d455-4999-94cd-da635509fe8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg"
Jul 20 23:02:24.218210 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:24.218181    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg"
Jul 20 23:02:24.346597 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:24.346566    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg"]
Jul 20 23:02:24.349359 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:02:24.349334    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be3e3ec_d455_4999_94cd_da635509fe8f.slice/crio-7e2b61e3b89db6260b4d86599f49a026dcc938ddd840389ac0b6e8e530029c6d WatchSource:0}: Error finding container 7e2b61e3b89db6260b4d86599f49a026dcc938ddd840389ac0b6e8e530029c6d: Status 404 returned error can't find the container with id 7e2b61e3b89db6260b4d86599f49a026dcc938ddd840389ac0b6e8e530029c6d
Jul 20 23:02:24.429460 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:24.429433    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:24.530158 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:24.530077    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:25.211240 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:25.211162    2550 generic.go:356] "Generic (PLEG): container finished" podID="0be3e3ec-d455-4999-94cd-da635509fe8f" containerID="e6d2ae480e4edb741eaae156b847ad964ec5594396a171b20d838c9dc8cbcb1e" exitCode=0
Jul 20 23:02:25.211240 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:25.211225    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg" event={"ID":"0be3e3ec-d455-4999-94cd-da635509fe8f","Type":"ContainerDied","Data":"e6d2ae480e4edb741eaae156b847ad964ec5594396a171b20d838c9dc8cbcb1e"}
Jul 20 23:02:25.211414 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:25.211262    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg" event={"ID":"0be3e3ec-d455-4999-94cd-da635509fe8f","Type":"ContainerStarted","Data":"7e2b61e3b89db6260b4d86599f49a026dcc938ddd840389ac0b6e8e530029c6d"}
Jul 20 23:02:26.346129 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:26.346106    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg"
Jul 20 23:02:26.521688 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:26.521602    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be3e3ec-d455-4999-94cd-da635509fe8f-maas-api-service-ca\") pod \"0be3e3ec-d455-4999-94cd-da635509fe8f\" (UID: \"0be3e3ec-d455-4999-94cd-da635509fe8f\") "
Jul 20 23:02:26.521963 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:26.521940    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be3e3ec-d455-4999-94cd-da635509fe8f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0be3e3ec-d455-4999-94cd-da635509fe8f" (UID: "0be3e3ec-d455-4999-94cd-da635509fe8f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 23:02:26.622157 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:26.622123    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be3e3ec-d455-4999-94cd-da635509fe8f-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:02:27.221691 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:27.221652    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg" event={"ID":"0be3e3ec-d455-4999-94cd-da635509fe8f","Type":"ContainerDied","Data":"7e2b61e3b89db6260b4d86599f49a026dcc938ddd840389ac0b6e8e530029c6d"}
Jul 20 23:02:27.221887 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:27.221665    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg"
Jul 20 23:02:27.221887 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:27.221769    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e2b61e3b89db6260b4d86599f49a026dcc938ddd840389ac0b6e8e530029c6d"
Jul 20 23:02:39.518593 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:39.518540    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:43.979945 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:43.979910    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg"]
Jul 20 23:02:43.982050 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:43.982024    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aae0e4-r7svg"]
Jul 20 23:02:43.989209 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:43.989186    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:44.365795 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:44.365708    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:02:44.368662 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:44.368637    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:44.845979 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:44.845945    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0be3e3ec-d455-4999-94cd-da635509fe8f" path="/var/lib/kubelet/pods/0be3e3ec-d455-4999-94cd-da635509fe8f/volumes"
Jul 20 23:02:44.954949 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:44.954919    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 23:02:44.957461 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:44.957437    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 23:02:50.100586 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:50.100548    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:50.187597 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:50.187563    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:50.430464 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:50.430363    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:50.627407 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:50.627367    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:50.952854 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:50.952821    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:51.254212 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:51.254135    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:51.519317 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:51.519227    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:54.025887 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:54.025838    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:54.108041 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:54.108002    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:54.175197 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:54.175162    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:54.512151 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:54.512115    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:54.780469 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:54.780375    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:54.881180 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:54.881138    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:58.530379 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:58.530341    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:58.842352 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:58.842258    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:58.966127 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:58.966088    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:02:59.139032 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:59.138942    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:02:59.496828 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:02:59.496770    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:02.074456 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:02.074362    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:02.196168 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:02.196123    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:02.291264 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:02.291229    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:02.628922 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:02.628885    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:02.775381 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:02.775347    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:02.876489 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:02.876448    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:06.546269 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:06.546229    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:07.778584 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:07.778551    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:03:12.060224 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:12.060183    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:12.160508 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:12.160471    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:15.613075 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:15.613042    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:15.714140 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:15.714104    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:23.567486 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:23.567442    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:23.668022 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:23.667986    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:27.494497 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.494460    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"]
Jul 20 23:03:27.494876 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.494799    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0be3e3ec-d455-4999-94cd-da635509fe8f" containerName="revoke-keys"
Jul 20 23:03:27.494876 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.494811    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0be3e3ec-d455-4999-94cd-da635509fe8f" containerName="revoke-keys"
Jul 20 23:03:27.494876 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.494877    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0be3e3ec-d455-4999-94cd-da635509fe8f" containerName="revoke-keys"
Jul 20 23:03:27.497680 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.497661    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.499674 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.499653    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-717ec3\"/\"openshift-service-ca.crt\""
Jul 20 23:03:27.500616 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.500596    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-717ec3\"/\"default-dockercfg-cplw4\""
Jul 20 23:03:27.500700 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.500599    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-717ec3\"/\"kube-root-ca.crt\""
Jul 20 23:03:27.500700 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.500607    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-717ec3\"/\"auth-test-model-717ec3-kserve-self-signed-certs\""
Jul 20 23:03:27.507413 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.507059    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"]
Jul 20 23:03:27.518466 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.518438    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:03:27.609776 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.609737    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-model-cache\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.609956 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.609780    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-home\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.609956 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.609817    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-dshm\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.609956 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.609860    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-tls-certs\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.609956 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.609890    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc2qb\" (UniqueName: \"kubernetes.io/projected/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-kube-api-access-cc2qb\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.609956 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.609918    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-tmp-dir\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.711189 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.711153    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-model-cache\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.711189 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.711188    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-home\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.711436 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.711207    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-dshm\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.711436 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.711235    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-tls-certs\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.711436 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.711260    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc2qb\" (UniqueName: \"kubernetes.io/projected/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-kube-api-access-cc2qb\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.711436 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.711286    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-tmp-dir\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.711691 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.711667    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-home\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.711787 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.711685    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-model-cache\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.711787 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.711760    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-tmp-dir\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.713576 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.713545    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-dshm\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.713802 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.713783    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-tls-certs\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.718805 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.718781    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc2qb\" (UniqueName: \"kubernetes.io/projected/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-kube-api-access-cc2qb\") pod \"auth-test-model-717ec3-kserve-5fd54c79-tnm2k\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") " pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.808619 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.808545    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:27.936258 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:27.936225    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"]
Jul 20 23:03:27.939828 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:03:27.939800    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod209bb941_33ad_4bd4_b3a5_c9a6791e4df3.slice/crio-06274dd2c52689e0aa6ffa0f4ba4f98431fa2a72ca3962e75aa31bcdaf9c0994 WatchSource:0}: Error finding container 06274dd2c52689e0aa6ffa0f4ba4f98431fa2a72ca3962e75aa31bcdaf9c0994: Status 404 returned error can't find the container with id 06274dd2c52689e0aa6ffa0f4ba4f98431fa2a72ca3962e75aa31bcdaf9c0994
Jul 20 23:03:28.518504 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:28.518465    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k" event={"ID":"209bb941-33ad-4bd4-b3a5-c9a6791e4df3","Type":"ContainerStarted","Data":"1d7957f06ef958bf4797efb53d988951944eced37a4d100f93e49add0e8bd3dc"}
Jul 20 23:03:28.518504 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:28.518505    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k" event={"ID":"209bb941-33ad-4bd4-b3a5-c9a6791e4df3","Type":"ContainerStarted","Data":"06274dd2c52689e0aa6ffa0f4ba4f98431fa2a72ca3962e75aa31bcdaf9c0994"}
Jul 20 23:03:28.536165 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:28.536104    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k" podStartSLOduration=1.536076327 podStartE2EDuration="1.536076327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 23:03:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 23:03:28.534907323 +0000 UTC m=+2444.312611378" watchObservedRunningTime="2026-07-20 23:03:28.536076327 +0000 UTC m=+2444.313780380"
Jul 20 23:03:31.164598 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:31.164554    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:03:37.809200 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:37.809154    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:37.809673 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:37.809213    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:37.821701 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:37.821674    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:38.580106 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:38.580069    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:03:45.068789 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:03:45.068758    2550 scope.go:117] "RemoveContainer" containerID="15cdbeb67dfb8ff03d9f72ef9afb0a1fb684b5b074d5184b2d0d47f26b92c217"
Jul 20 23:04:07.097502 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.097459    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"]
Jul 20 23:04:07.100882 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.100858    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.103105 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.103081    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1dd4fb\"/\"openshift-service-ca.crt\""
Jul 20 23:04:07.103232 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.103081    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1dd4fb\"/\"kube-root-ca.crt\""
Jul 20 23:04:07.103900 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.103883    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1dd4fb\"/\"default-dockercfg-46t7l\""
Jul 20 23:04:07.104005 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.103955    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1dd4fb\"/\"auth-test-model-1dd4fb-kserve-self-signed-certs\""
Jul 20 23:04:07.113117 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.113091    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"]
Jul 20 23:04:07.133771 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.133739    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:04:07.162363 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.162331    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-tls-certs\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.162523 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.162372    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-dshm\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.162523 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.162387    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-tmp-dir\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.162523 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.162428    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96nwg\" (UniqueName: \"kubernetes.io/projected/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-kube-api-access-96nwg\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.162687 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.162526    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-model-cache\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.162687 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.162605    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-home\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.263882 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.263836    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-home\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.264074 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.263889    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-tls-certs\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.264074 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.263921    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-dshm\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.264074 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.263939    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-tmp-dir\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.264074 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.263977    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96nwg\" (UniqueName: \"kubernetes.io/projected/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-kube-api-access-96nwg\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.264074 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.264022    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-model-cache\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.264355 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.264317    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-home\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.264355 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.264336    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-tmp-dir\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.264488 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.264431    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-model-cache\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.266229 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.266207    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-dshm\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.266499 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.266474    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-tls-certs\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.271424 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.271400    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96nwg\" (UniqueName: \"kubernetes.io/projected/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-kube-api-access-96nwg\") pod \"auth-test-model-1dd4fb-kserve-986897587-5l8lk\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") " pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.412075 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.411963    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:07.545929 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.545903    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"]
Jul 20 23:04:07.548189 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:04:07.548161    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod308e1d71_6ce6_4ca7_9e7c_6d4ecf4feba2.slice/crio-bb19a1a8b2ebf06054c5b71ce1a323ecb1b54fbbcd0d300017e954d3b60de693 WatchSource:0}: Error finding container bb19a1a8b2ebf06054c5b71ce1a323ecb1b54fbbcd0d300017e954d3b60de693: Status 404 returned error can't find the container with id bb19a1a8b2ebf06054c5b71ce1a323ecb1b54fbbcd0d300017e954d3b60de693
Jul 20 23:04:07.710864 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.710829    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk" event={"ID":"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2","Type":"ContainerStarted","Data":"bd747fb68ad017f01fef9216d640bb8613fffec66be744f450596809200aa1fb"}
Jul 20 23:04:07.711040 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.710876    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk" event={"ID":"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2","Type":"ContainerStarted","Data":"bb19a1a8b2ebf06054c5b71ce1a323ecb1b54fbbcd0d300017e954d3b60de693"}
Jul 20 23:04:07.726931 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:07.726879    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk" podStartSLOduration=0.726863837 podStartE2EDuration="726.863837ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 23:04:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 23:04:07.726251109 +0000 UTC m=+2483.503955160" watchObservedRunningTime="2026-07-20 23:04:07.726863837 +0000 UTC m=+2483.504567890"
Jul 20 23:04:10.762445 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:10.762411    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:04:17.412791 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:17.412754    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:17.413282 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:17.412813    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:17.424893 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:17.424868    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:17.770587 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:17.770561    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:04:24.465477 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:24.465435    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:04:39.661409 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:39.661329    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:04:58.166402 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:04:58.166365    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:05:05.373041 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:05:05.373006    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:05:05.425544 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:05:05.425509    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:05:14.773208 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:05:14.773165    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:05:35.467661 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:05:35.467623    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:05:40.179381 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:05:40.179339    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:05:54.063567 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:05:54.063529    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:06:08.785349 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:08.785267    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:06:13.612987 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:13.612952    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"]
Jul 20 23:06:13.638707 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:13.638681    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:06:14.393857 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:14.393814    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"]
Jul 20 23:06:14.963253 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:14.963218    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:06:20.679329 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:20.679294    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:06:24.863251 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:24.863201    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:06:28.632455 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.632406    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k" podUID="209bb941-33ad-4bd4-b3a5-c9a6791e4df3" containerName="main" containerID="cri-o://1d7957f06ef958bf4797efb53d988951944eced37a4d100f93e49add0e8bd3dc" gracePeriod=45
Jul 20 23:06:28.886167 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.886101    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:06:28.965840 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.965803    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-tmp-dir\") pod \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") "
Jul 20 23:06:28.966032 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.965859    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-dshm\") pod \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") "
Jul 20 23:06:28.966032 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.965884    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-tls-certs\") pod \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") "
Jul 20 23:06:28.966032 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.965927    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc2qb\" (UniqueName: \"kubernetes.io/projected/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-kube-api-access-cc2qb\") pod \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") "
Jul 20 23:06:28.966032 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.965958    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-home\") pod \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") "
Jul 20 23:06:28.966032 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.966018    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-model-cache\") pod \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\" (UID: \"209bb941-33ad-4bd4-b3a5-c9a6791e4df3\") "
Jul 20 23:06:28.966287 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.966109    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "209bb941-33ad-4bd4-b3a5-c9a6791e4df3" (UID: "209bb941-33ad-4bd4-b3a5-c9a6791e4df3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:06:28.966287 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.966267    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-home" (OuterVolumeSpecName: "home") pod "209bb941-33ad-4bd4-b3a5-c9a6791e4df3" (UID: "209bb941-33ad-4bd4-b3a5-c9a6791e4df3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:06:28.966401 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.966286    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-model-cache" (OuterVolumeSpecName: "model-cache") pod "209bb941-33ad-4bd4-b3a5-c9a6791e4df3" (UID: "209bb941-33ad-4bd4-b3a5-c9a6791e4df3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:06:28.966401 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.966327    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-tmp-dir\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:06:28.968234 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.968210    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-dshm" (OuterVolumeSpecName: "dshm") pod "209bb941-33ad-4bd4-b3a5-c9a6791e4df3" (UID: "209bb941-33ad-4bd4-b3a5-c9a6791e4df3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:06:28.968341 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.968233    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-kube-api-access-cc2qb" (OuterVolumeSpecName: "kube-api-access-cc2qb") pod "209bb941-33ad-4bd4-b3a5-c9a6791e4df3" (UID: "209bb941-33ad-4bd4-b3a5-c9a6791e4df3"). InnerVolumeSpecName "kube-api-access-cc2qb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 23:06:28.968341 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:28.968292    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "209bb941-33ad-4bd4-b3a5-c9a6791e4df3" (UID: "209bb941-33ad-4bd4-b3a5-c9a6791e4df3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 23:06:29.067338 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.067301    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-home\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:06:29.067338 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.067338    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-model-cache\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:06:29.067529 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.067352    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-dshm\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:06:29.067529 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.067360    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-tls-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:06:29.067529 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.067369    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc2qb\" (UniqueName: \"kubernetes.io/projected/209bb941-33ad-4bd4-b3a5-c9a6791e4df3-kube-api-access-cc2qb\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:06:29.389474 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.389435    2550 generic.go:356] "Generic (PLEG): container finished" podID="209bb941-33ad-4bd4-b3a5-c9a6791e4df3" containerID="1d7957f06ef958bf4797efb53d988951944eced37a4d100f93e49add0e8bd3dc" exitCode=0
Jul 20 23:06:29.389474 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.389476    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"
Jul 20 23:06:29.389683 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.389523    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k" event={"ID":"209bb941-33ad-4bd4-b3a5-c9a6791e4df3","Type":"ContainerDied","Data":"1d7957f06ef958bf4797efb53d988951944eced37a4d100f93e49add0e8bd3dc"}
Jul 20 23:06:29.389683 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.389562    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k" event={"ID":"209bb941-33ad-4bd4-b3a5-c9a6791e4df3","Type":"ContainerDied","Data":"06274dd2c52689e0aa6ffa0f4ba4f98431fa2a72ca3962e75aa31bcdaf9c0994"}
Jul 20 23:06:29.389683 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.389582    2550 scope.go:117] "RemoveContainer" containerID="1d7957f06ef958bf4797efb53d988951944eced37a4d100f93e49add0e8bd3dc"
Jul 20 23:06:29.398238 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.398220    2550 scope.go:117] "RemoveContainer" containerID="1d7957f06ef958bf4797efb53d988951944eced37a4d100f93e49add0e8bd3dc"
Jul 20 23:06:29.398488 ip-10-0-129-16 kubenswrapper[2550]: E0720 23:06:29.398469    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d7957f06ef958bf4797efb53d988951944eced37a4d100f93e49add0e8bd3dc\": container with ID starting with 1d7957f06ef958bf4797efb53d988951944eced37a4d100f93e49add0e8bd3dc not found: ID does not exist" containerID="1d7957f06ef958bf4797efb53d988951944eced37a4d100f93e49add0e8bd3dc"
Jul 20 23:06:29.398536 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.398497    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7957f06ef958bf4797efb53d988951944eced37a4d100f93e49add0e8bd3dc"} err="failed to get container status \"1d7957f06ef958bf4797efb53d988951944eced37a4d100f93e49add0e8bd3dc\": rpc error: code = NotFound desc = could not find container \"1d7957f06ef958bf4797efb53d988951944eced37a4d100f93e49add0e8bd3dc\": container with ID starting with 1d7957f06ef958bf4797efb53d988951944eced37a4d100f93e49add0e8bd3dc not found: ID does not exist"
Jul 20 23:06:29.410302 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.410276    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"]
Jul 20 23:06:29.413829 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.413799    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk" podUID="308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2" containerName="main" containerID="cri-o://bd747fb68ad017f01fef9216d640bb8613fffec66be744f450596809200aa1fb" gracePeriod=45
Jul 20 23:06:29.414140 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.414087    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-717ec3/auth-test-model-717ec3-kserve-5fd54c79-tnm2k"]
Jul 20 23:06:29.660067 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.660037    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:06:29.772561 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.772520    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-dshm\") pod \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") "
Jul 20 23:06:29.772786 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.772575    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-tmp-dir\") pod \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") "
Jul 20 23:06:29.772786 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.772630    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-tls-certs\") pod \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") "
Jul 20 23:06:29.772786 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.772669    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96nwg\" (UniqueName: \"kubernetes.io/projected/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-kube-api-access-96nwg\") pod \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") "
Jul 20 23:06:29.772786 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.772696    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-model-cache\") pod \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") "
Jul 20 23:06:29.772786 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.772763    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-home\") pod \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\" (UID: \"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2\") "
Jul 20 23:06:29.773070 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.772955    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2" (UID: "308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:06:29.773070 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.773053    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-model-cache" (OuterVolumeSpecName: "model-cache") pod "308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2" (UID: "308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:06:29.773143 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.773080    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-home" (OuterVolumeSpecName: "home") pod "308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2" (UID: "308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:06:29.774944 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.774921    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-dshm" (OuterVolumeSpecName: "dshm") pod "308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2" (UID: "308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:06:29.774944 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.774937    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-kube-api-access-96nwg" (OuterVolumeSpecName: "kube-api-access-96nwg") pod "308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2" (UID: "308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2"). InnerVolumeSpecName "kube-api-access-96nwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 23:06:29.775104 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.774956    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2" (UID: "308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 23:06:29.874298 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.874260    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-tls-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:06:29.874298 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.874292    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96nwg\" (UniqueName: \"kubernetes.io/projected/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-kube-api-access-96nwg\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:06:29.874495 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.874304    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-model-cache\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:06:29.874495 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.874335    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-home\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:06:29.874495 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.874343    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-dshm\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:06:29.874495 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:29.874351    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2-tmp-dir\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:06:30.396506 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:30.396468    2550 generic.go:356] "Generic (PLEG): container finished" podID="308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2" containerID="bd747fb68ad017f01fef9216d640bb8613fffec66be744f450596809200aa1fb" exitCode=0
Jul 20 23:06:30.396681 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:30.396511    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"
Jul 20 23:06:30.396681 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:30.396535    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk" event={"ID":"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2","Type":"ContainerDied","Data":"bd747fb68ad017f01fef9216d640bb8613fffec66be744f450596809200aa1fb"}
Jul 20 23:06:30.396681 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:30.396572    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk" event={"ID":"308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2","Type":"ContainerDied","Data":"bb19a1a8b2ebf06054c5b71ce1a323ecb1b54fbbcd0d300017e954d3b60de693"}
Jul 20 23:06:30.396681 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:30.396592    2550 scope.go:117] "RemoveContainer" containerID="bd747fb68ad017f01fef9216d640bb8613fffec66be744f450596809200aa1fb"
Jul 20 23:06:30.405393 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:30.405375    2550 scope.go:117] "RemoveContainer" containerID="bd747fb68ad017f01fef9216d640bb8613fffec66be744f450596809200aa1fb"
Jul 20 23:06:30.405649 ip-10-0-129-16 kubenswrapper[2550]: E0720 23:06:30.405629    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd747fb68ad017f01fef9216d640bb8613fffec66be744f450596809200aa1fb\": container with ID starting with bd747fb68ad017f01fef9216d640bb8613fffec66be744f450596809200aa1fb not found: ID does not exist" containerID="bd747fb68ad017f01fef9216d640bb8613fffec66be744f450596809200aa1fb"
Jul 20 23:06:30.405711 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:30.405657    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd747fb68ad017f01fef9216d640bb8613fffec66be744f450596809200aa1fb"} err="failed to get container status \"bd747fb68ad017f01fef9216d640bb8613fffec66be744f450596809200aa1fb\": rpc error: code = NotFound desc = could not find container \"bd747fb68ad017f01fef9216d640bb8613fffec66be744f450596809200aa1fb\": container with ID starting with bd747fb68ad017f01fef9216d640bb8613fffec66be744f450596809200aa1fb not found: ID does not exist"
Jul 20 23:06:30.419145 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:30.419121    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"]
Jul 20 23:06:30.424925 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:30.424903    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/auth-test-model-1dd4fb-kserve-986897587-5l8lk"]
Jul 20 23:06:30.845864 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:30.845828    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="209bb941-33ad-4bd4-b3a5-c9a6791e4df3" path="/var/lib/kubelet/pods/209bb941-33ad-4bd4-b3a5-c9a6791e4df3/volumes"
Jul 20 23:06:30.846240 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:30.846225    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2" path="/var/lib/kubelet/pods/308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2/volumes"
Jul 20 23:06:45.103947 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:45.103913    2550 scope.go:117] "RemoveContainer" containerID="bcc38cfd965bdf1bf167249aaf44a696bb7060a230fa41c7dd267900eac69102"
Jul 20 23:06:45.111902 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:45.111884    2550 scope.go:117] "RemoveContainer" containerID="aecdb32f423203f281300b0e04cf7fde31219c0f846ae05fc3ad5deb748aaad3"
Jul 20 23:06:50.794652 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.794609    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"]
Jul 20 23:06:50.795061 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.795040    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2" containerName="main"
Jul 20 23:06:50.795061 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.795055    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2" containerName="main"
Jul 20 23:06:50.795137 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.795068    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="209bb941-33ad-4bd4-b3a5-c9a6791e4df3" containerName="main"
Jul 20 23:06:50.795137 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.795074    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="209bb941-33ad-4bd4-b3a5-c9a6791e4df3" containerName="main"
Jul 20 23:06:50.795137 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.795133    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="308e1d71-6ce6-4ca7-9e7c-6d4ecf4feba2" containerName="main"
Jul 20 23:06:50.795227 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.795144    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="209bb941-33ad-4bd4-b3a5-c9a6791e4df3" containerName="main"
Jul 20 23:06:50.800019 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.799997    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:50.802460 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.802436    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1dd4fb\"/\"openshift-service-ca.crt\""
Jul 20 23:06:50.802599 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.802474    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1dd4fb\"/\"kube-root-ca.crt\""
Jul 20 23:06:50.803012 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.802995    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1dd4fb\"/\"sub-test-model-1dd4fb-kserve-self-signed-certs\""
Jul 20 23:06:50.803093 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.803017    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1dd4fb\"/\"default-dockercfg-46t7l\""
Jul 20 23:06:50.808267 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.808241    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"]
Jul 20 23:06:50.820544 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.820518    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:06:50.948401 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.948359    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-home\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:50.948559 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.948407    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-model-cache\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:50.948559 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.948427    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-tmp-dir\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:50.948559 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.948531    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wcp8\" (UniqueName: \"kubernetes.io/projected/769aa08e-e101-4376-907e-5f0f782bd790-kube-api-access-7wcp8\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:50.948665 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.948576    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/769aa08e-e101-4376-907e-5f0f782bd790-tls-certs\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:50.948665 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:50.948610    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-dshm\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.049810 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.049701    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wcp8\" (UniqueName: \"kubernetes.io/projected/769aa08e-e101-4376-907e-5f0f782bd790-kube-api-access-7wcp8\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.049810 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.049769    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/769aa08e-e101-4376-907e-5f0f782bd790-tls-certs\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.049810 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.049793    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-dshm\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.050043 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.049864    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-home\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.050043 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.049898    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-model-cache\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.050043 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.050016    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-tmp-dir\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.050317 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.050298    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-home\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.050376 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.050344    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-model-cache\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.050424 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.050409    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-tmp-dir\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.052174 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.052156    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-dshm\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.052360 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.052343    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/769aa08e-e101-4376-907e-5f0f782bd790-tls-certs\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.058254 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.058234    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wcp8\" (UniqueName: \"kubernetes.io/projected/769aa08e-e101-4376-907e-5f0f782bd790-kube-api-access-7wcp8\") pod \"sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") " pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.109278 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.109238    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:06:51.242929 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.242901    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"]
Jul 20 23:06:51.245146 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:06:51.245118    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod769aa08e_e101_4376_907e_5f0f782bd790.slice/crio-9262189dfbb2db31e1c3f31adbf4386316371657955a96e317b9785204529288 WatchSource:0}: Error finding container 9262189dfbb2db31e1c3f31adbf4386316371657955a96e317b9785204529288: Status 404 returned error can't find the container with id 9262189dfbb2db31e1c3f31adbf4386316371657955a96e317b9785204529288
Jul 20 23:06:51.500390 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.500350    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6" event={"ID":"769aa08e-e101-4376-907e-5f0f782bd790","Type":"ContainerStarted","Data":"a2246a49766c444cb55740a532fc7553ab1797dc50d6d0c4aca11d1043b9d76e"}
Jul 20 23:06:51.500582 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.500401    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6" event={"ID":"769aa08e-e101-4376-907e-5f0f782bd790","Type":"ContainerStarted","Data":"9262189dfbb2db31e1c3f31adbf4386316371657955a96e317b9785204529288"}
Jul 20 23:06:51.519230 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:51.519171    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6" podStartSLOduration=1.5191561519999999 podStartE2EDuration="1.519156152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 23:06:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 23:06:51.517062734 +0000 UTC m=+2647.294766785" watchObservedRunningTime="2026-07-20 23:06:51.519156152 +0000 UTC m=+2647.296860203"
Jul 20 23:06:54.684294 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:06:54.684251    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:07:01.109619 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:01.109579    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:07:01.110106 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:01.109636    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:07:01.122579 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:01.122548    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:07:01.565562 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:01.565533    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:07:08.674519 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:08.674477    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:07:24.874792 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:24.874753    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:07:30.965543 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:30.965507    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:07:31.783371 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:31.783336    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"]
Jul 20 23:07:31.806695 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:31.806620    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:07:40.667285 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:40.667195    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:07:44.984946 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:44.984915    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 23:07:44.987223 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:44.987193    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 23:07:45.139698 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:45.139660    2550 scope.go:117] "RemoveContainer" containerID="1efff9403f3449b9500b7d9b586caa30a9bd1b2b9bffff03f396a6585328417e"
Jul 20 23:07:45.147730 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:45.147702    2550 scope.go:117] "RemoveContainer" containerID="34ed1fad1b69e8d2e1e04e9b2227167baff70f0abf60495d7c9bcb58ce6a13ee"
Jul 20 23:07:46.803655 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:46.803616    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6" podUID="769aa08e-e101-4376-907e-5f0f782bd790" containerName="main" containerID="cri-o://a2246a49766c444cb55740a532fc7553ab1797dc50d6d0c4aca11d1043b9d76e" gracePeriod=45
Jul 20 23:07:47.059518 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.059456    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:07:47.138429 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.138390    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/769aa08e-e101-4376-907e-5f0f782bd790-tls-certs\") pod \"769aa08e-e101-4376-907e-5f0f782bd790\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") "
Jul 20 23:07:47.138612 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.138460    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-tmp-dir\") pod \"769aa08e-e101-4376-907e-5f0f782bd790\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") "
Jul 20 23:07:47.138612 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.138496    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-home\") pod \"769aa08e-e101-4376-907e-5f0f782bd790\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") "
Jul 20 23:07:47.138612 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.138519    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-model-cache\") pod \"769aa08e-e101-4376-907e-5f0f782bd790\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") "
Jul 20 23:07:47.138612 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.138543    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-dshm\") pod \"769aa08e-e101-4376-907e-5f0f782bd790\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") "
Jul 20 23:07:47.138612 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.138609    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wcp8\" (UniqueName: \"kubernetes.io/projected/769aa08e-e101-4376-907e-5f0f782bd790-kube-api-access-7wcp8\") pod \"769aa08e-e101-4376-907e-5f0f782bd790\" (UID: \"769aa08e-e101-4376-907e-5f0f782bd790\") "
Jul 20 23:07:47.138913 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.138788    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-model-cache" (OuterVolumeSpecName: "model-cache") pod "769aa08e-e101-4376-907e-5f0f782bd790" (UID: "769aa08e-e101-4376-907e-5f0f782bd790"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:07:47.138913 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.138795    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "769aa08e-e101-4376-907e-5f0f782bd790" (UID: "769aa08e-e101-4376-907e-5f0f782bd790"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:07:47.138913 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.138810    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-home" (OuterVolumeSpecName: "home") pod "769aa08e-e101-4376-907e-5f0f782bd790" (UID: "769aa08e-e101-4376-907e-5f0f782bd790"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:07:47.139041 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.138936    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-tmp-dir\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:07:47.139041 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.138951    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-home\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:07:47.139041 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.138969    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-model-cache\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:07:47.140701 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.140658    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/769aa08e-e101-4376-907e-5f0f782bd790-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "769aa08e-e101-4376-907e-5f0f782bd790" (UID: "769aa08e-e101-4376-907e-5f0f782bd790"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 23:07:47.140829 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.140769    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-dshm" (OuterVolumeSpecName: "dshm") pod "769aa08e-e101-4376-907e-5f0f782bd790" (UID: "769aa08e-e101-4376-907e-5f0f782bd790"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:07:47.140829 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.140798    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/769aa08e-e101-4376-907e-5f0f782bd790-kube-api-access-7wcp8" (OuterVolumeSpecName: "kube-api-access-7wcp8") pod "769aa08e-e101-4376-907e-5f0f782bd790" (UID: "769aa08e-e101-4376-907e-5f0f782bd790"). InnerVolumeSpecName "kube-api-access-7wcp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 23:07:47.240247 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.240213    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wcp8\" (UniqueName: \"kubernetes.io/projected/769aa08e-e101-4376-907e-5f0f782bd790-kube-api-access-7wcp8\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:07:47.240247 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.240243    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/769aa08e-e101-4376-907e-5f0f782bd790-tls-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:07:47.240247 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.240254    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/769aa08e-e101-4376-907e-5f0f782bd790-dshm\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:07:47.772692 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.772653    2550 generic.go:356] "Generic (PLEG): container finished" podID="769aa08e-e101-4376-907e-5f0f782bd790" containerID="a2246a49766c444cb55740a532fc7553ab1797dc50d6d0c4aca11d1043b9d76e" exitCode=0
Jul 20 23:07:47.772692 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.772696    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"
Jul 20 23:07:47.772929 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.772756    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6" event={"ID":"769aa08e-e101-4376-907e-5f0f782bd790","Type":"ContainerDied","Data":"a2246a49766c444cb55740a532fc7553ab1797dc50d6d0c4aca11d1043b9d76e"}
Jul 20 23:07:47.772929 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.772806    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6" event={"ID":"769aa08e-e101-4376-907e-5f0f782bd790","Type":"ContainerDied","Data":"9262189dfbb2db31e1c3f31adbf4386316371657955a96e317b9785204529288"}
Jul 20 23:07:47.772929 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.772827    2550 scope.go:117] "RemoveContainer" containerID="a2246a49766c444cb55740a532fc7553ab1797dc50d6d0c4aca11d1043b9d76e"
Jul 20 23:07:47.781788 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.781770    2550 scope.go:117] "RemoveContainer" containerID="a2246a49766c444cb55740a532fc7553ab1797dc50d6d0c4aca11d1043b9d76e"
Jul 20 23:07:47.782031 ip-10-0-129-16 kubenswrapper[2550]: E0720 23:07:47.782011    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2246a49766c444cb55740a532fc7553ab1797dc50d6d0c4aca11d1043b9d76e\": container with ID starting with a2246a49766c444cb55740a532fc7553ab1797dc50d6d0c4aca11d1043b9d76e not found: ID does not exist" containerID="a2246a49766c444cb55740a532fc7553ab1797dc50d6d0c4aca11d1043b9d76e"
Jul 20 23:07:47.782108 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.782039    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2246a49766c444cb55740a532fc7553ab1797dc50d6d0c4aca11d1043b9d76e"} err="failed to get container status \"a2246a49766c444cb55740a532fc7553ab1797dc50d6d0c4aca11d1043b9d76e\": rpc error: code = NotFound desc = could not find container \"a2246a49766c444cb55740a532fc7553ab1797dc50d6d0c4aca11d1043b9d76e\": container with ID starting with a2246a49766c444cb55740a532fc7553ab1797dc50d6d0c4aca11d1043b9d76e not found: ID does not exist"
Jul 20 23:07:47.794163 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.794135    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"]
Jul 20 23:07:47.795907 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:47.795887    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/sub-test-model-1dd4fb-kserve-5785bfbb97-c8hs6"]
Jul 20 23:07:48.844593 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:48.844559    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="769aa08e-e101-4376-907e-5f0f782bd790" path="/var/lib/kubelet/pods/769aa08e-e101-4376-907e-5f0f782bd790/volumes"
Jul 20 23:07:56.003034 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.002997    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"]
Jul 20 23:07:56.003658 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.003344    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="769aa08e-e101-4376-907e-5f0f782bd790" containerName="main"
Jul 20 23:07:56.003658 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.003357    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="769aa08e-e101-4376-907e-5f0f782bd790" containerName="main"
Jul 20 23:07:56.003658 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.003408    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="769aa08e-e101-4376-907e-5f0f782bd790" containerName="main"
Jul 20 23:07:56.011200 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.011168    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.015246 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.015209    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-717ec3\"/\"kube-root-ca.crt\""
Jul 20 23:07:56.015844 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.015815    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"]
Jul 20 23:07:56.016055 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.016008    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-717ec3\"/\"openshift-service-ca.crt\""
Jul 20 23:07:56.016143 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.016069    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-717ec3\"/\"rate-test-model-717ec3-kserve-self-signed-certs\""
Jul 20 23:07:56.016405 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.016347    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-717ec3\"/\"default-dockercfg-cplw4\""
Jul 20 23:07:56.037100 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.037071    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:07:56.112615 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.112554    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-tmp-dir\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.112840 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.112631    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-tls-certs\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.112840 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.112677    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-home\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.112840 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.112706    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-model-cache\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.112840 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.112802    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-dshm\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.112986 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.112867    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq2qf\" (UniqueName: \"kubernetes.io/projected/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-kube-api-access-dq2qf\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.214165 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.214120    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-tmp-dir\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.214165 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.214174    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-tls-certs\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.214395 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.214198    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-home\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.214395 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.214221    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-model-cache\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.214395 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.214249    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-dshm\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.214395 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.214277    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq2qf\" (UniqueName: \"kubernetes.io/projected/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-kube-api-access-dq2qf\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.214640 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.214612    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-tmp-dir\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.214710 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.214628    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-model-cache\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.214710 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.214678    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-home\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.216549 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.216524    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-dshm\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.216662 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.216633    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-tls-certs\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.222668 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.222646    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq2qf\" (UniqueName: \"kubernetes.io/projected/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-kube-api-access-dq2qf\") pod \"rate-test-model-717ec3-kserve-6df77f4859-g47tc\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") " pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.323893 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.323805    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:07:56.457839 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.457810    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"]
Jul 20 23:07:56.460003 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:07:56.459973    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76aabb4c_94b4_44ad_9c6c_ee6c06bdf8af.slice/crio-715c17c9c1bc0fca76945b1257074b0eba9e4494fdd31769ccd0228400e1ff7a WatchSource:0}: Error finding container 715c17c9c1bc0fca76945b1257074b0eba9e4494fdd31769ccd0228400e1ff7a: Status 404 returned error can't find the container with id 715c17c9c1bc0fca76945b1257074b0eba9e4494fdd31769ccd0228400e1ff7a
Jul 20 23:07:56.818690 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.818649    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc" event={"ID":"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af","Type":"ContainerStarted","Data":"187e5eb05bb70dd26e4c773eafce38c81850771bfef8312416459c2059939231"}
Jul 20 23:07:56.818900 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.818702    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc" event={"ID":"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af","Type":"ContainerStarted","Data":"715c17c9c1bc0fca76945b1257074b0eba9e4494fdd31769ccd0228400e1ff7a"}
Jul 20 23:07:56.835105 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:56.835035    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc" podStartSLOduration=1.835018831 podStartE2EDuration="1.835018831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 23:07:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 23:07:56.833895242 +0000 UTC m=+2712.611599294" watchObservedRunningTime="2026-07-20 23:07:56.835018831 +0000 UTC m=+2712.612723020"
Jul 20 23:07:59.370709 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:07:59.370670    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:08:06.324880 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:06.324830    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:08:06.325307 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:06.324899    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:08:06.337624 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:06.337586    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:08:06.877440 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:06.877400    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:08:12.863219 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:12.863181    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:08:12.965942 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:12.965899    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:08:13.590207 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.590174    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"]
Jul 20 23:08:13.598257 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.598228    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.601630 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.601598    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1dd4fb\"/\"rate-test-model-1dd4fb-kserve-self-signed-certs\""
Jul 20 23:08:13.601819 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.601750    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1dd4fb\"/\"default-dockercfg-46t7l\""
Jul 20 23:08:13.602287 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.602265    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1dd4fb\"/\"openshift-service-ca.crt\""
Jul 20 23:08:13.602930 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.602758    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1dd4fb\"/\"kube-root-ca.crt\""
Jul 20 23:08:13.604269 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.604245    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"]
Jul 20 23:08:13.616120 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.616096    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:08:13.665388 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.665349    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-tmp-dir\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.665585 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.665399    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-home\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.665585 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.665417    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-dshm\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.665585 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.665498    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b981c30-1e59-433d-855d-45db2191a1f4-tls-certs\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.665585 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.665571    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-model-cache\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.665800 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.665626    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxdnn\" (UniqueName: \"kubernetes.io/projected/1b981c30-1e59-433d-855d-45db2191a1f4-kube-api-access-qxdnn\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.766280 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.766241    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-model-cache\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.766479 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.766296    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxdnn\" (UniqueName: \"kubernetes.io/projected/1b981c30-1e59-433d-855d-45db2191a1f4-kube-api-access-qxdnn\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.766479 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.766332    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-tmp-dir\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.766479 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.766372    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-home\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.766479 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.766391    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-dshm\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.766479 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.766428    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b981c30-1e59-433d-855d-45db2191a1f4-tls-certs\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.766791 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.766762    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-model-cache\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.766862 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.766787    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-home\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.766862 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.766797    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-tmp-dir\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.768689 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.768667    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-dshm\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.768973 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.768956    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b981c30-1e59-433d-855d-45db2191a1f4-tls-certs\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.774290 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.774270    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxdnn\" (UniqueName: \"kubernetes.io/projected/1b981c30-1e59-433d-855d-45db2191a1f4-kube-api-access-qxdnn\") pod \"rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") " pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:13.909943 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:13.909841    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:14.041943 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:14.041905    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"]
Jul 20 23:08:14.044373 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:08:14.044336    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b981c30_1e59_433d_855d_45db2191a1f4.slice/crio-84fbad06d7d3a08c5a54608806ea2b39c8264898fca0b615e8c26aab11d66b99 WatchSource:0}: Error finding container 84fbad06d7d3a08c5a54608806ea2b39c8264898fca0b615e8c26aab11d66b99: Status 404 returned error can't find the container with id 84fbad06d7d3a08c5a54608806ea2b39c8264898fca0b615e8c26aab11d66b99
Jul 20 23:08:14.904222 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:14.904178    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn" event={"ID":"1b981c30-1e59-433d-855d-45db2191a1f4","Type":"ContainerStarted","Data":"2972d51c52c6c88d6bcebea4dd7ea5518861118a6a20abf75b0fbb811defd2d6"}
Jul 20 23:08:14.904399 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:14.904232    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn" event={"ID":"1b981c30-1e59-433d-855d-45db2191a1f4","Type":"ContainerStarted","Data":"84fbad06d7d3a08c5a54608806ea2b39c8264898fca0b615e8c26aab11d66b99"}
Jul 20 23:08:14.920050 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:14.919767    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn" podStartSLOduration=1.919747553 podStartE2EDuration="1.919747553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 23:08:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 23:08:14.919415801 +0000 UTC m=+2730.697119865" watchObservedRunningTime="2026-07-20 23:08:14.919747553 +0000 UTC m=+2730.697451606"
Jul 20 23:08:16.470000 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:16.469964    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:08:16.571039 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:16.571003    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:08:17.172438 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:17.172399    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:08:23.910349 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:23.910254    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:23.910349 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:23.910310    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:23.923019 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:23.922994    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:23.957275 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:23.957246    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:08:24.340226 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:24.340195    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:08:24.439895 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:24.439859    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:08:28.663814 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:28.663771    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:08:44.232225 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:44.232184    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:08:45.168926 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:45.168893    2550 scope.go:117] "RemoveContainer" containerID="e6d2ae480e4edb741eaae156b847ad964ec5594396a171b20d838c9dc8cbcb1e"
Jul 20 23:08:45.176665 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:45.176641    2550 scope.go:117] "RemoveContainer" containerID="f7d5f26e7235e922d87a2b0680303f73713974dd91b544d0c94e9a4618c9e9c8"
Jul 20 23:08:46.563836 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:46.563802    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:08:51.230512 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:51.230475    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"]
Jul 20 23:08:51.253228 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:51.253194    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:08:52.083821 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:52.083784    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"]
Jul 20 23:08:52.104649 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:08:52.104616    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:01.872433 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:01.872386    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:09:06.253072 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.253032    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc" podUID="76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af" containerName="main" containerID="cri-o://187e5eb05bb70dd26e4c773eafce38c81850771bfef8312416459c2059939231" gracePeriod=45
Jul 20 23:09:06.507522 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.507467    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:09:06.622975 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.622898    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-dshm\") pod \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") "
Jul 20 23:09:06.622975 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.622946    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-tls-certs\") pod \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") "
Jul 20 23:09:06.623187 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.623001    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dq2qf\" (UniqueName: \"kubernetes.io/projected/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-kube-api-access-dq2qf\") pod \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") "
Jul 20 23:09:06.623187 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.623028    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-tmp-dir\") pod \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") "
Jul 20 23:09:06.623187 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.623054    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-model-cache\") pod \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") "
Jul 20 23:09:06.623330 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.623200    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-home\") pod \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\" (UID: \"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af\") "
Jul 20 23:09:06.623330 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.623309    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-model-cache" (OuterVolumeSpecName: "model-cache") pod "76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af" (UID: "76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:09:06.623444 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.623323    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af" (UID: "76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:09:06.623517 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.623487    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-home" (OuterVolumeSpecName: "home") pod "76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af" (UID: "76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:09:06.623564 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.623526    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-tmp-dir\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:06.623564 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.623547    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-model-cache\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:06.625089 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.625061    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-dshm" (OuterVolumeSpecName: "dshm") pod "76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af" (UID: "76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:09:06.625089 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.625082    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af" (UID: "76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 23:09:06.625242 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.625124    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-kube-api-access-dq2qf" (OuterVolumeSpecName: "kube-api-access-dq2qf") pod "76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af" (UID: "76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af"). InnerVolumeSpecName "kube-api-access-dq2qf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 23:09:06.724066 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.724021    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-home\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:06.724066 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.724060    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-dshm\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:06.724066 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.724069    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-tls-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:06.724066 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:06.724081    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dq2qf\" (UniqueName: \"kubernetes.io/projected/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af-kube-api-access-dq2qf\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:07.104039 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.103934    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn" podUID="1b981c30-1e59-433d-855d-45db2191a1f4" containerName="main" containerID="cri-o://2972d51c52c6c88d6bcebea4dd7ea5518861118a6a20abf75b0fbb811defd2d6" gracePeriod=45
Jul 20 23:09:07.137373 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.137341    2550 generic.go:356] "Generic (PLEG): container finished" podID="76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af" containerID="187e5eb05bb70dd26e4c773eafce38c81850771bfef8312416459c2059939231" exitCode=0
Jul 20 23:09:07.137493 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.137384    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"
Jul 20 23:09:07.137493 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.137383    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc" event={"ID":"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af","Type":"ContainerDied","Data":"187e5eb05bb70dd26e4c773eafce38c81850771bfef8312416459c2059939231"}
Jul 20 23:09:07.137569 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.137506    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc" event={"ID":"76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af","Type":"ContainerDied","Data":"715c17c9c1bc0fca76945b1257074b0eba9e4494fdd31769ccd0228400e1ff7a"}
Jul 20 23:09:07.137569 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.137531    2550 scope.go:117] "RemoveContainer" containerID="187e5eb05bb70dd26e4c773eafce38c81850771bfef8312416459c2059939231"
Jul 20 23:09:07.176882 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.176853    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"]
Jul 20 23:09:07.180931 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.180906    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-717ec3/rate-test-model-717ec3-kserve-6df77f4859-g47tc"]
Jul 20 23:09:07.214126 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.214106    2550 scope.go:117] "RemoveContainer" containerID="187e5eb05bb70dd26e4c773eafce38c81850771bfef8312416459c2059939231"
Jul 20 23:09:07.214426 ip-10-0-129-16 kubenswrapper[2550]: E0720 23:09:07.214409    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"187e5eb05bb70dd26e4c773eafce38c81850771bfef8312416459c2059939231\": container with ID starting with 187e5eb05bb70dd26e4c773eafce38c81850771bfef8312416459c2059939231 not found: ID does not exist" containerID="187e5eb05bb70dd26e4c773eafce38c81850771bfef8312416459c2059939231"
Jul 20 23:09:07.214488 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.214438    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"187e5eb05bb70dd26e4c773eafce38c81850771bfef8312416459c2059939231"} err="failed to get container status \"187e5eb05bb70dd26e4c773eafce38c81850771bfef8312416459c2059939231\": rpc error: code = NotFound desc = could not find container \"187e5eb05bb70dd26e4c773eafce38c81850771bfef8312416459c2059939231\": container with ID starting with 187e5eb05bb70dd26e4c773eafce38c81850771bfef8312416459c2059939231 not found: ID does not exist"
Jul 20 23:09:07.354588 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.354527    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:09:07.430100 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.430064    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-home\") pod \"1b981c30-1e59-433d-855d-45db2191a1f4\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") "
Jul 20 23:09:07.430100 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.430111    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-model-cache\") pod \"1b981c30-1e59-433d-855d-45db2191a1f4\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") "
Jul 20 23:09:07.430365 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.430156    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-tmp-dir\") pod \"1b981c30-1e59-433d-855d-45db2191a1f4\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") "
Jul 20 23:09:07.430365 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.430198    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b981c30-1e59-433d-855d-45db2191a1f4-tls-certs\") pod \"1b981c30-1e59-433d-855d-45db2191a1f4\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") "
Jul 20 23:09:07.430365 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.430239    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-dshm\") pod \"1b981c30-1e59-433d-855d-45db2191a1f4\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") "
Jul 20 23:09:07.430365 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.430269    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxdnn\" (UniqueName: \"kubernetes.io/projected/1b981c30-1e59-433d-855d-45db2191a1f4-kube-api-access-qxdnn\") pod \"1b981c30-1e59-433d-855d-45db2191a1f4\" (UID: \"1b981c30-1e59-433d-855d-45db2191a1f4\") "
Jul 20 23:09:07.430527 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.430377    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-home" (OuterVolumeSpecName: "home") pod "1b981c30-1e59-433d-855d-45db2191a1f4" (UID: "1b981c30-1e59-433d-855d-45db2191a1f4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:09:07.430527 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.430445    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-model-cache" (OuterVolumeSpecName: "model-cache") pod "1b981c30-1e59-433d-855d-45db2191a1f4" (UID: "1b981c30-1e59-433d-855d-45db2191a1f4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:09:07.430527 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.430455    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1b981c30-1e59-433d-855d-45db2191a1f4" (UID: "1b981c30-1e59-433d-855d-45db2191a1f4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:09:07.430639 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.430555    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-tmp-dir\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:07.430639 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.430573    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-home\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:07.430639 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.430585    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-model-cache\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:07.432380 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.432350    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-dshm" (OuterVolumeSpecName: "dshm") pod "1b981c30-1e59-433d-855d-45db2191a1f4" (UID: "1b981c30-1e59-433d-855d-45db2191a1f4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 23:09:07.432494 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.432383    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b981c30-1e59-433d-855d-45db2191a1f4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1b981c30-1e59-433d-855d-45db2191a1f4" (UID: "1b981c30-1e59-433d-855d-45db2191a1f4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 23:09:07.432494 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.432394    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b981c30-1e59-433d-855d-45db2191a1f4-kube-api-access-qxdnn" (OuterVolumeSpecName: "kube-api-access-qxdnn") pod "1b981c30-1e59-433d-855d-45db2191a1f4" (UID: "1b981c30-1e59-433d-855d-45db2191a1f4"). InnerVolumeSpecName "kube-api-access-qxdnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 23:09:07.531747 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.531696    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b981c30-1e59-433d-855d-45db2191a1f4-tls-certs\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:07.531747 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.531743    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b981c30-1e59-433d-855d-45db2191a1f4-dshm\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:07.531747 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:07.531753    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxdnn\" (UniqueName: \"kubernetes.io/projected/1b981c30-1e59-433d-855d-45db2191a1f4-kube-api-access-qxdnn\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:08.102275 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.102241    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7702-140185560324256"]
Jul 20 23:09:08.102568 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.102556    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b981c30-1e59-433d-855d-45db2191a1f4" containerName="main"
Jul 20 23:09:08.102621 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.102570    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b981c30-1e59-433d-855d-45db2191a1f4" containerName="main"
Jul 20 23:09:08.102621 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.102584    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af" containerName="main"
Jul 20 23:09:08.102621 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.102589    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af" containerName="main"
Jul 20 23:09:08.102749 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.102655    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b981c30-1e59-433d-855d-45db2191a1f4" containerName="main"
Jul 20 23:09:08.102749 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.102661    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af" containerName="main"
Jul 20 23:09:08.105520 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.105503    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7702-140185560324256"
Jul 20 23:09:08.107869 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.107851    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wsblt\""
Jul 20 23:09:08.110645 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.110623    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7702-140185560324256"]
Jul 20 23:09:08.136766 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.136745    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmhhv\" (UniqueName: \"kubernetes.io/projected/3f013bf1-187d-4a67-a873-ef4a73cbe8bf-kube-api-access-bmhhv\") pod \"test-curl-7702-140185560324256\" (UID: \"3f013bf1-187d-4a67-a873-ef4a73cbe8bf\") " pod="opendatahub/test-curl-7702-140185560324256"
Jul 20 23:09:08.143133 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.143112    2550 generic.go:356] "Generic (PLEG): container finished" podID="1b981c30-1e59-433d-855d-45db2191a1f4" containerID="2972d51c52c6c88d6bcebea4dd7ea5518861118a6a20abf75b0fbb811defd2d6" exitCode=0
Jul 20 23:09:08.143247 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.143152    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"
Jul 20 23:09:08.143247 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.143201    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn" event={"ID":"1b981c30-1e59-433d-855d-45db2191a1f4","Type":"ContainerDied","Data":"2972d51c52c6c88d6bcebea4dd7ea5518861118a6a20abf75b0fbb811defd2d6"}
Jul 20 23:09:08.143247 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.143234    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn" event={"ID":"1b981c30-1e59-433d-855d-45db2191a1f4","Type":"ContainerDied","Data":"84fbad06d7d3a08c5a54608806ea2b39c8264898fca0b615e8c26aab11d66b99"}
Jul 20 23:09:08.143377 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.143251    2550 scope.go:117] "RemoveContainer" containerID="2972d51c52c6c88d6bcebea4dd7ea5518861118a6a20abf75b0fbb811defd2d6"
Jul 20 23:09:08.151412 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.151391    2550 scope.go:117] "RemoveContainer" containerID="2972d51c52c6c88d6bcebea4dd7ea5518861118a6a20abf75b0fbb811defd2d6"
Jul 20 23:09:08.151694 ip-10-0-129-16 kubenswrapper[2550]: E0720 23:09:08.151676    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2972d51c52c6c88d6bcebea4dd7ea5518861118a6a20abf75b0fbb811defd2d6\": container with ID starting with 2972d51c52c6c88d6bcebea4dd7ea5518861118a6a20abf75b0fbb811defd2d6 not found: ID does not exist" containerID="2972d51c52c6c88d6bcebea4dd7ea5518861118a6a20abf75b0fbb811defd2d6"
Jul 20 23:09:08.151774 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.151701    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2972d51c52c6c88d6bcebea4dd7ea5518861118a6a20abf75b0fbb811defd2d6"} err="failed to get container status \"2972d51c52c6c88d6bcebea4dd7ea5518861118a6a20abf75b0fbb811defd2d6\": rpc error: code = NotFound desc = could not find container \"2972d51c52c6c88d6bcebea4dd7ea5518861118a6a20abf75b0fbb811defd2d6\": container with ID starting with 2972d51c52c6c88d6bcebea4dd7ea5518861118a6a20abf75b0fbb811defd2d6 not found: ID does not exist"
Jul 20 23:09:08.163902 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.163881    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"]
Jul 20 23:09:08.167612 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.167592    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1dd4fb/rate-test-model-1dd4fb-kserve-b7f7d548c-xdmmn"]
Jul 20 23:09:08.237850 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.237818    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmhhv\" (UniqueName: \"kubernetes.io/projected/3f013bf1-187d-4a67-a873-ef4a73cbe8bf-kube-api-access-bmhhv\") pod \"test-curl-7702-140185560324256\" (UID: \"3f013bf1-187d-4a67-a873-ef4a73cbe8bf\") " pod="opendatahub/test-curl-7702-140185560324256"
Jul 20 23:09:08.247483 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.247463    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmhhv\" (UniqueName: \"kubernetes.io/projected/3f013bf1-187d-4a67-a873-ef4a73cbe8bf-kube-api-access-bmhhv\") pod \"test-curl-7702-140185560324256\" (UID: \"3f013bf1-187d-4a67-a873-ef4a73cbe8bf\") " pod="opendatahub/test-curl-7702-140185560324256"
Jul 20 23:09:08.415954 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.415858    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7702-140185560324256"
Jul 20 23:09:08.543572 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.543540    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7702-140185560324256"]
Jul 20 23:09:08.546795 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:09:08.546761    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f013bf1_187d_4a67_a873_ef4a73cbe8bf.slice/crio-836cf18d30ed9545f723422c46cdbee38c4008ad51ee67af7931540cabcf4a69 WatchSource:0}: Error finding container 836cf18d30ed9545f723422c46cdbee38c4008ad51ee67af7931540cabcf4a69: Status 404 returned error can't find the container with id 836cf18d30ed9545f723422c46cdbee38c4008ad51ee67af7931540cabcf4a69
Jul 20 23:09:08.548000 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.547983    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 23:09:08.845525 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.845493    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b981c30-1e59-433d-855d-45db2191a1f4" path="/var/lib/kubelet/pods/1b981c30-1e59-433d-855d-45db2191a1f4/volumes"
Jul 20 23:09:08.845920 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:08.845904    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af" path="/var/lib/kubelet/pods/76aabb4c-94b4-44ad-9c6c-ee6c06bdf8af/volumes"
Jul 20 23:09:09.152342 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:09.152262    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7702-140185560324256" event={"ID":"3f013bf1-187d-4a67-a873-ef4a73cbe8bf","Type":"ContainerStarted","Data":"836cf18d30ed9545f723422c46cdbee38c4008ad51ee67af7931540cabcf4a69"}
Jul 20 23:09:10.158298 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:10.158262    2550 generic.go:356] "Generic (PLEG): container finished" podID="3f013bf1-187d-4a67-a873-ef4a73cbe8bf" containerID="d972127ed47679b185c6b90a471f7a707e072c6a972554130d3710b7fc153af4" exitCode=0
Jul 20 23:09:10.158680 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:10.158349    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7702-140185560324256" event={"ID":"3f013bf1-187d-4a67-a873-ef4a73cbe8bf","Type":"ContainerDied","Data":"d972127ed47679b185c6b90a471f7a707e072c6a972554130d3710b7fc153af4"}
Jul 20 23:09:11.293022 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.292999    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7702-140185560324256"
Jul 20 23:09:11.311320 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.311292    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7702-140185560324256_3f013bf1-187d-4a67-a873-ef4a73cbe8bf/test-curl-7702-140185560324256/0.log"
Jul 20 23:09:11.334540 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.334511    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7702-140185560324256"]
Jul 20 23:09:11.337946 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.337923    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7702-140185560324256"]
Jul 20 23:09:11.361640 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.361615    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmhhv\" (UniqueName: \"kubernetes.io/projected/3f013bf1-187d-4a67-a873-ef4a73cbe8bf-kube-api-access-bmhhv\") pod \"3f013bf1-187d-4a67-a873-ef4a73cbe8bf\" (UID: \"3f013bf1-187d-4a67-a873-ef4a73cbe8bf\") "
Jul 20 23:09:11.363784 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.363759    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f013bf1-187d-4a67-a873-ef4a73cbe8bf-kube-api-access-bmhhv" (OuterVolumeSpecName: "kube-api-access-bmhhv") pod "3f013bf1-187d-4a67-a873-ef4a73cbe8bf" (UID: "3f013bf1-187d-4a67-a873-ef4a73cbe8bf"). InnerVolumeSpecName "kube-api-access-bmhhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 23:09:11.463105 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.463081    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmhhv\" (UniqueName: \"kubernetes.io/projected/3f013bf1-187d-4a67-a873-ef4a73cbe8bf-kube-api-access-bmhhv\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:11.518487 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.518458    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv"]
Jul 20 23:09:11.518830 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.518816    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f013bf1-187d-4a67-a873-ef4a73cbe8bf" containerName="test-curl-7702-140185560324256"
Jul 20 23:09:11.518890 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.518832    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f013bf1-187d-4a67-a873-ef4a73cbe8bf" containerName="test-curl-7702-140185560324256"
Jul 20 23:09:11.518890 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.518888    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f013bf1-187d-4a67-a873-ef4a73cbe8bf" containerName="test-curl-7702-140185560324256"
Jul 20 23:09:11.521888 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.521871    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv"
Jul 20 23:09:11.523871 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.523845    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 23:09:11.523962 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.523857    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4f4tw\""
Jul 20 23:09:11.529107 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.529082    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv"]
Jul 20 23:09:11.563571 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.563539    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c63626b0-03fe-4fb2-a254-f80c3f559e36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv\" (UID: \"c63626b0-03fe-4fb2-a254-f80c3f559e36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv"
Jul 20 23:09:11.664952 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.664911    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c63626b0-03fe-4fb2-a254-f80c3f559e36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv\" (UID: \"c63626b0-03fe-4fb2-a254-f80c3f559e36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv"
Jul 20 23:09:11.665545 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.665524    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c63626b0-03fe-4fb2-a254-f80c3f559e36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv\" (UID: \"c63626b0-03fe-4fb2-a254-f80c3f559e36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv"
Jul 20 23:09:11.833279 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.833183    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv"
Jul 20 23:09:11.964536 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:11.964511    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv"]
Jul 20 23:09:11.966285 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:09:11.966260    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63626b0_03fe_4fb2_a254_f80c3f559e36.slice/crio-6d86b179c95251a84cccb8f9d9e419f5b3bd71143fb50220ce9c4757b6d9e670 WatchSource:0}: Error finding container 6d86b179c95251a84cccb8f9d9e419f5b3bd71143fb50220ce9c4757b6d9e670: Status 404 returned error can't find the container with id 6d86b179c95251a84cccb8f9d9e419f5b3bd71143fb50220ce9c4757b6d9e670
Jul 20 23:09:12.016085 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:12.016061    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:12.116014 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:12.115916    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:12.169415 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:12.169380    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv" event={"ID":"c63626b0-03fe-4fb2-a254-f80c3f559e36","Type":"ContainerStarted","Data":"11dd091b67aa28c5eefc6aeb380b82f59d5b61a8e3bdfd5360f99ec7789d7070"}
Jul 20 23:09:12.169549 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:12.169443    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv" event={"ID":"c63626b0-03fe-4fb2-a254-f80c3f559e36","Type":"ContainerStarted","Data":"6d86b179c95251a84cccb8f9d9e419f5b3bd71143fb50220ce9c4757b6d9e670"}
Jul 20 23:09:12.170598 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:12.170575    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="836cf18d30ed9545f723422c46cdbee38c4008ad51ee67af7931540cabcf4a69"
Jul 20 23:09:12.170598 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:12.170592    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7702-140185560324256"
Jul 20 23:09:12.187451 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:12.187411    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv" podStartSLOduration=1.187398024 podStartE2EDuration="1.187398024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 23:09:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 23:09:12.186281698 +0000 UTC m=+2787.963985761" watchObservedRunningTime="2026-07-20 23:09:12.187398024 +0000 UTC m=+2787.965102054"
Jul 20 23:09:12.188033 ip-10-0-129-16 kubenswrapper[2550]: E0720 23:09:12.188002    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7702-140185560324256\" is forbidden: User \"system:node:ip-10-0-129-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-16.ec2.internal' and this object" podUID="3f013bf1-187d-4a67-a873-ef4a73cbe8bf" pod="opendatahub/test-curl-7702-140185560324256"
Jul 20 23:09:12.844064 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:12.843990    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f013bf1-187d-4a67-a873-ef4a73cbe8bf" path="/var/lib/kubelet/pods/3f013bf1-187d-4a67-a873-ef4a73cbe8bf/volumes"
Jul 20 23:09:13.176171 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:13.176086    2550 generic.go:356] "Generic (PLEG): container finished" podID="c63626b0-03fe-4fb2-a254-f80c3f559e36" containerID="11dd091b67aa28c5eefc6aeb380b82f59d5b61a8e3bdfd5360f99ec7789d7070" exitCode=0
Jul 20 23:09:13.176171 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:13.176152    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv" event={"ID":"c63626b0-03fe-4fb2-a254-f80c3f559e36","Type":"ContainerDied","Data":"11dd091b67aa28c5eefc6aeb380b82f59d5b61a8e3bdfd5360f99ec7789d7070"}
Jul 20 23:09:14.311383 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:14.311359    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv"
Jul 20 23:09:14.387604 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:14.387569    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c63626b0-03fe-4fb2-a254-f80c3f559e36-maas-api-service-ca\") pod \"c63626b0-03fe-4fb2-a254-f80c3f559e36\" (UID: \"c63626b0-03fe-4fb2-a254-f80c3f559e36\") "
Jul 20 23:09:14.387951 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:14.387930    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c63626b0-03fe-4fb2-a254-f80c3f559e36-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c63626b0-03fe-4fb2-a254-f80c3f559e36" (UID: "c63626b0-03fe-4fb2-a254-f80c3f559e36"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 23:09:14.488803 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:14.488773    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c63626b0-03fe-4fb2-a254-f80c3f559e36-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:15.186990 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:15.186901    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv"
Jul 20 23:09:15.186990 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:15.186917    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv" event={"ID":"c63626b0-03fe-4fb2-a254-f80c3f559e36","Type":"ContainerDied","Data":"6d86b179c95251a84cccb8f9d9e419f5b3bd71143fb50220ce9c4757b6d9e670"}
Jul 20 23:09:15.186990 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:15.186973    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d86b179c95251a84cccb8f9d9e419f5b3bd71143fb50220ce9c4757b6d9e670"
Jul 20 23:09:21.976179 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:21.976145    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:25.162192 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:25.162149    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:09:26.593277 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:26.593242    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:26.615734 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:26.615684    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv"]
Jul 20 23:09:26.618045 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:26.618016    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-717ec3-wfqwv"]
Jul 20 23:09:26.844025 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:26.843947    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c63626b0-03fe-4fb2-a254-f80c3f559e36" path="/var/lib/kubelet/pods/c63626b0-03fe-4fb2-a254-f80c3f559e36/volumes"
Jul 20 23:09:27.013064 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:27.013008    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:33.093405 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.093363    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:33.202518 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.202471    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:33.375476 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.375395    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:33.622277 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.622239    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:33.734255 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.734225    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8"]
Jul 20 23:09:33.734564 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.734552    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c63626b0-03fe-4fb2-a254-f80c3f559e36" containerName="revoke-keys"
Jul 20 23:09:33.734607 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.734565    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c63626b0-03fe-4fb2-a254-f80c3f559e36" containerName="revoke-keys"
Jul 20 23:09:33.734641 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.734620    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c63626b0-03fe-4fb2-a254-f80c3f559e36" containerName="revoke-keys"
Jul 20 23:09:33.737736 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.737704    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8"
Jul 20 23:09:33.740437 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.740417    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4f4tw\""
Jul 20 23:09:33.740997 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.740981    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 23:09:33.745820 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.745789    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8"]
Jul 20 23:09:33.847585 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.847556    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8b1b05-88d3-4730-ac1b-5259031dac70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8\" (UID: \"da8b1b05-88d3-4730-ac1b-5259031dac70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8"
Jul 20 23:09:33.948731 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.948686    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8b1b05-88d3-4730-ac1b-5259031dac70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8\" (UID: \"da8b1b05-88d3-4730-ac1b-5259031dac70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8"
Jul 20 23:09:33.949282 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:33.949264    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8b1b05-88d3-4730-ac1b-5259031dac70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8\" (UID: \"da8b1b05-88d3-4730-ac1b-5259031dac70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8"
Jul 20 23:09:34.048302 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:34.048244    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8"
Jul 20 23:09:34.176810 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:34.176784    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8"]
Jul 20 23:09:34.179071 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:09:34.179046    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda8b1b05_88d3_4730_ac1b_5259031dac70.slice/crio-b89b302f2402704ce2038d63a601c2ccfc1de6e01350fd99c08161ba4789da85 WatchSource:0}: Error finding container b89b302f2402704ce2038d63a601c2ccfc1de6e01350fd99c08161ba4789da85: Status 404 returned error can't find the container with id b89b302f2402704ce2038d63a601c2ccfc1de6e01350fd99c08161ba4789da85
Jul 20 23:09:34.251136 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:34.251108    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:34.275938 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:34.275915    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8" event={"ID":"da8b1b05-88d3-4730-ac1b-5259031dac70","Type":"ContainerStarted","Data":"b89b302f2402704ce2038d63a601c2ccfc1de6e01350fd99c08161ba4789da85"}
Jul 20 23:09:34.351303 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:34.351282    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:35.281815 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:35.281781    2550 generic.go:356] "Generic (PLEG): container finished" podID="da8b1b05-88d3-4730-ac1b-5259031dac70" containerID="c45e87d6ebda2ca643333f9cf5e2e7d94f8477ecfeac6b53a24c6d56ac5891b2" exitCode=0
Jul 20 23:09:35.282199 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:35.281877    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8" event={"ID":"da8b1b05-88d3-4730-ac1b-5259031dac70","Type":"ContainerDied","Data":"c45e87d6ebda2ca643333f9cf5e2e7d94f8477ecfeac6b53a24c6d56ac5891b2"}
Jul 20 23:09:36.417861 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:36.417822    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8"
Jul 20 23:09:36.572303 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:36.572207    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8b1b05-88d3-4730-ac1b-5259031dac70-maas-api-service-ca\") pod \"da8b1b05-88d3-4730-ac1b-5259031dac70\" (UID: \"da8b1b05-88d3-4730-ac1b-5259031dac70\") "
Jul 20 23:09:36.572610 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:36.572589    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da8b1b05-88d3-4730-ac1b-5259031dac70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "da8b1b05-88d3-4730-ac1b-5259031dac70" (UID: "da8b1b05-88d3-4730-ac1b-5259031dac70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 23:09:36.673650 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:36.673612    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da8b1b05-88d3-4730-ac1b-5259031dac70-maas-api-service-ca\") on node \"ip-10-0-129-16.ec2.internal\" DevicePath \"\""
Jul 20 23:09:37.292710 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:37.292683    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8"
Jul 20 23:09:37.292886 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:37.292736    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8" event={"ID":"da8b1b05-88d3-4730-ac1b-5259031dac70","Type":"ContainerDied","Data":"b89b302f2402704ce2038d63a601c2ccfc1de6e01350fd99c08161ba4789da85"}
Jul 20 23:09:37.292886 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:37.292785    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b89b302f2402704ce2038d63a601c2ccfc1de6e01350fd99c08161ba4789da85"
Jul 20 23:09:44.148339 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:44.148307    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:46.890982 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:46.890943    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p9k95"]
Jul 20 23:09:48.785706 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:48.785664    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:48.789098 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:48.789069    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8"]
Jul 20 23:09:48.791989 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:48.791965    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1dd4fb-skxd8"]
Jul 20 23:09:48.844190 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:48.844163    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da8b1b05-88d3-4730-ac1b-5259031dac70" path="/var/lib/kubelet/pods/da8b1b05-88d3-4730-ac1b-5259031dac70/volumes"
Jul 20 23:09:49.167981 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:49.167897    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:55.042582 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:55.042550    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:55.109744 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:55.109697    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:55.294551 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:55.294480    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:09:55.509216 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:09:55.509183    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:10:01.290140 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:01.290111    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-f4fmx_417e7a1c-68f2-4f80-8eca-0b89c8bf3186/manager/0.log"
Jul 20 23:10:01.402879 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:01.402846    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vk7tj_6f2c38fb-6cc5-438b-b824-a243ca239ad1/manager/0.log"
Jul 20 23:10:01.773083 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:01.773053    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-q56kq_58ae02e3-3d12-451f-b178-5f7ea84f259c/manager/0.log"
Jul 20 23:10:02.131282 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:02.131203    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b4989886d-xc2dt_71554333-2c49-4889-b65b-08b6af2a1c43/copy-manifests/0.log"
Jul 20 23:10:02.138158 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:02.138129    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b4989886d-xc2dt_71554333-2c49-4889-b65b-08b6af2a1c43/manager/0.log"
Jul 20 23:10:02.694487 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:02.694446    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743140-jb4ws_29edf151-bdba-45e6-a271-6763690c79be/cleanup/0.log"
Jul 20 23:10:04.291369 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:04.291332    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ldbtt_02e0c2f7-9618-4c6f-bfb8-cec0553c0c9c/manager/0.log"
Jul 20 23:10:04.735913 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:04.735889    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-p9k95_195cf9ea-d5a7-4cd9-a08c-405bd180186c/limitador/0.log"
Jul 20 23:10:04.856491 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:04.856461    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-6fq76_8399921c-0a40-4f17-8797-85a780ad4432/manager/0.log"
Jul 20 23:10:05.307701 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:05.307673    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ltpmz_e0931d18-25b6-425b-acd7-075c46f058dc/discovery/0.log"
Jul 20 23:10:05.528216 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:05.528189    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f85576d5-mqcrz_3b6f9220-0c28-42f7-9d81-9d0eadcf9958/kube-auth-proxy/0.log"
Jul 20 23:10:05.981402 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:05.981371    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54dbf84666-xw84d_2e863cc6-8c15-4725-a3b5-9748719edb3c/router/0.log"
Jul 20 23:10:06.773378 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:06.773331    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-w9wq8_b0d5a181-bf1f-47af-929e-cdab9a8d0d80/main/0.log"
Jul 20 23:10:12.417199 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:12.417158    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:10:12.518148 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:12.518118    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:10:15.143655 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:15.143622    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-89dp7_d7df0b06-2bc3-45b8-8443-6c4583b2ca28/global-pull-secret-syncer/0.log"
Jul 20 23:10:15.225612 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:15.225584    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log"
Jul 20 23:10:15.304446 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:15.304421    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-16.ec2.internal_3b8d2414fddfe8167462c801d0be4b0d/haproxy/0.log"
Jul 20 23:10:20.990286 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:20.990256    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ldbtt_02e0c2f7-9618-4c6f-bfb8-cec0553c0c9c/manager/0.log"
Jul 20 23:10:21.220191 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:21.220158    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-p9k95_195cf9ea-d5a7-4cd9-a08c-405bd180186c/limitador/0.log"
Jul 20 23:10:21.289653 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:21.289571    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-6fq76_8399921c-0a40-4f17-8797-85a780ad4432/manager/0.log"
Jul 20 23:10:23.108845 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:23.108815    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-9mdsd_e9e5fca5-4619-48ab-b7a7-51ac2743a899/cluster-monitoring-operator/0.log"
Jul 20 23:10:23.338255 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:23.338224    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gpkdp_421fb845-fa15-4321-8774-7661b458d95d/node-exporter/0.log"
Jul 20 23:10:23.368698 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:23.368632    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gpkdp_421fb845-fa15-4321-8774-7661b458d95d/kube-rbac-proxy/0.log"
Jul 20 23:10:23.388697 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:23.388672    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gpkdp_421fb845-fa15-4321-8774-7661b458d95d/init-textfile/0.log"
Jul 20 23:10:25.174769 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.174729    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"]
Jul 20 23:10:25.175249 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.175227    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da8b1b05-88d3-4730-ac1b-5259031dac70" containerName="revoke-keys"
Jul 20 23:10:25.175328 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.175253    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="da8b1b05-88d3-4730-ac1b-5259031dac70" containerName="revoke-keys"
Jul 20 23:10:25.175380 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.175335    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="da8b1b05-88d3-4730-ac1b-5259031dac70" containerName="revoke-keys"
Jul 20 23:10:25.178345 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.178323    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.182076 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.182048    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-drpjk\"/\"kube-root-ca.crt\""
Jul 20 23:10:25.182199 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.182048    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-drpjk\"/\"openshift-service-ca.crt\""
Jul 20 23:10:25.182497 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.182481    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-drpjk\"/\"default-dockercfg-bdb5w\""
Jul 20 23:10:25.189283 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.189262    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"]
Jul 20 23:10:25.244320 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.244280    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7935933c-2d2d-4b9b-8154-4435e133659d-podres\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.244507 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.244325    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7935933c-2d2d-4b9b-8154-4435e133659d-proc\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.244507 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.244380    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7935933c-2d2d-4b9b-8154-4435e133659d-lib-modules\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.244507 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.244437    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb2pm\" (UniqueName: \"kubernetes.io/projected/7935933c-2d2d-4b9b-8154-4435e133659d-kube-api-access-sb2pm\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.244644 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.244532    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7935933c-2d2d-4b9b-8154-4435e133659d-sys\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.345962 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.345913    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7935933c-2d2d-4b9b-8154-4435e133659d-sys\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.345962 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.345972    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7935933c-2d2d-4b9b-8154-4435e133659d-podres\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.346221 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.345994    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7935933c-2d2d-4b9b-8154-4435e133659d-proc\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.346221 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.346015    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7935933c-2d2d-4b9b-8154-4435e133659d-lib-modules\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.346221 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.346047    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7935933c-2d2d-4b9b-8154-4435e133659d-sys\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.346221 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.346055    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb2pm\" (UniqueName: \"kubernetes.io/projected/7935933c-2d2d-4b9b-8154-4435e133659d-kube-api-access-sb2pm\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.346221 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.346118    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7935933c-2d2d-4b9b-8154-4435e133659d-proc\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.346221 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.346141    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7935933c-2d2d-4b9b-8154-4435e133659d-lib-modules\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.346221 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.346147    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7935933c-2d2d-4b9b-8154-4435e133659d-podres\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.353993 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.353968    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb2pm\" (UniqueName: \"kubernetes.io/projected/7935933c-2d2d-4b9b-8154-4435e133659d-kube-api-access-sb2pm\") pod \"perf-node-gather-daemonset-wp2gj\" (UID: \"7935933c-2d2d-4b9b-8154-4435e133659d\") " pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.488004 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.487969    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:25.620302 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:25.620276    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"]
Jul 20 23:10:25.622308 ip-10-0-129-16 kubenswrapper[2550]: W0720 23:10:25.622281    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7935933c_2d2d_4b9b_8154_4435e133659d.slice/crio-c66c758dadc7bed4c1b2a269070b8a6e23dd86c41393ff26c08512af19dc0c62 WatchSource:0}: Error finding container c66c758dadc7bed4c1b2a269070b8a6e23dd86c41393ff26c08512af19dc0c62: Status 404 returned error can't find the container with id c66c758dadc7bed4c1b2a269070b8a6e23dd86c41393ff26c08512af19dc0c62
Jul 20 23:10:26.507746 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:26.507687    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj" event={"ID":"7935933c-2d2d-4b9b-8154-4435e133659d","Type":"ContainerStarted","Data":"9b7b6a9bdb295bc45a237225c0dc4a72303abcd32518658852e03f6bee2fe2a4"}
Jul 20 23:10:26.508212 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:26.507769    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj" event={"ID":"7935933c-2d2d-4b9b-8154-4435e133659d","Type":"ContainerStarted","Data":"c66c758dadc7bed4c1b2a269070b8a6e23dd86c41393ff26c08512af19dc0c62"}
Jul 20 23:10:26.508212 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:26.507817    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:26.523066 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:26.523020    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj" podStartSLOduration=1.5230062100000001 podStartE2EDuration="1.52300621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 23:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 23:10:26.522364525 +0000 UTC m=+2862.300068577" watchObservedRunningTime="2026-07-20 23:10:26.52300621 +0000 UTC m=+2862.300710258"
Jul 20 23:10:26.950997 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:26.950971    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-7zqcq_db2efb42-daa5-4924-8c76-47f89a07bc7d/volume-data-source-validator/0.log"
Jul 20 23:10:28.067537 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:28.067509    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x6w7q_69c9b463-2c26-4b92-b415-c5bf20afb209/dns/0.log"
Jul 20 23:10:28.089394 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:28.089363    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x6w7q_69c9b463-2c26-4b92-b415-c5bf20afb209/kube-rbac-proxy/0.log"
Jul 20 23:10:28.110035 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:28.110009    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bw9fm_b94e391c-6611-4879-a037-77852e20a2af/dns-node-resolver/0.log"
Jul 20 23:10:28.681378 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:28.681349    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-79d9754994-wsl8w_5ec36065-a4fb-466a-9aa6-df816c8c8669/registry/0.log"
Jul 20 23:10:28.727979 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:28.727951    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7vxbl_07ea4c4b-623c-44c9-a630-22f2cf3f91f0/node-ca/0.log"
Jul 20 23:10:29.783500 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:29.783461    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ltpmz_e0931d18-25b6-425b-acd7-075c46f058dc/discovery/0.log"
Jul 20 23:10:29.829139 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:29.829109    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f85576d5-mqcrz_3b6f9220-0c28-42f7-9d81-9d0eadcf9958/kube-auth-proxy/0.log"
Jul 20 23:10:29.984056 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:29.984026    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54dbf84666-xw84d_2e863cc6-8c15-4725-a3b5-9748719edb3c/router/0.log"
Jul 20 23:10:30.646152 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:30.646121    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-548zv_c5302090-68d3-401b-b54f-dd5c292e7d10/serve-healthcheck-canary/0.log"
Jul 20 23:10:31.210367 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:31.210339    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dx845_fa7d4497-7ffe-4428-bb59-a6a68a97cfd6/kube-rbac-proxy/0.log"
Jul 20 23:10:31.230183 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:31.230150    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dx845_fa7d4497-7ffe-4428-bb59-a6a68a97cfd6/exporter/0.log"
Jul 20 23:10:31.251265 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:31.251235    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dx845_fa7d4497-7ffe-4428-bb59-a6a68a97cfd6/extractor/0.log"
Jul 20 23:10:32.522240 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:32.522213    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-drpjk/perf-node-gather-daemonset-wp2gj"
Jul 20 23:10:33.509376 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:33.509324    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-f4fmx_417e7a1c-68f2-4f80-8eca-0b89c8bf3186/manager/0.log"
Jul 20 23:10:33.691060 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:33.691019    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vk7tj_6f2c38fb-6cc5-438b-b824-a243ca239ad1/manager/0.log"
Jul 20 23:10:33.925558 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:33.925479    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-q56kq_58ae02e3-3d12-451f-b178-5f7ea84f259c/manager/0.log"
Jul 20 23:10:34.125639 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:34.125608    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b4989886d-xc2dt_71554333-2c49-4889-b65b-08b6af2a1c43/manager/0.log"
Jul 20 23:10:34.148331 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:34.148294    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b4989886d-xc2dt_71554333-2c49-4889-b65b-08b6af2a1c43/copy-manifests/0.log"
Jul 20 23:10:35.556187 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:35.556160    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ltpmz_e0931d18-25b6-425b-acd7-075c46f058dc/discovery/0.log"
Jul 20 23:10:35.584636 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:35.584560    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f85576d5-mqcrz_3b6f9220-0c28-42f7-9d81-9d0eadcf9958/kube-auth-proxy/0.log"
Jul 20 23:10:35.712606 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:35.712575    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54dbf84666-xw84d_2e863cc6-8c15-4725-a3b5-9748719edb3c/router/0.log"
Jul 20 23:10:37.102962 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:37.102930    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ltpmz_e0931d18-25b6-425b-acd7-075c46f058dc/discovery/0.log"
Jul 20 23:10:37.133157 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:37.133129    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f85576d5-mqcrz_3b6f9220-0c28-42f7-9d81-9d0eadcf9958/kube-auth-proxy/0.log"
Jul 20 23:10:37.252588 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:37.252552    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54dbf84666-xw84d_2e863cc6-8c15-4725-a3b5-9748719edb3c/router/0.log"
Jul 20 23:10:41.030069 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:41.030022    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-87h8h_22aad18a-8d9e-4509-9773-5906515b3782/kube-storage-version-migrator-operator/1.log"
Jul 20 23:10:41.030870 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:41.030850    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-87h8h_22aad18a-8d9e-4509-9773-5906515b3782/kube-storage-version-migrator-operator/0.log"
Jul 20 23:10:42.335007 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:42.334974    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g7tg5_e1c875ab-aa59-4235-ac20-201d722fcc7f/kube-multus-additional-cni-plugins/0.log"
Jul 20 23:10:42.356948 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:42.356882    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g7tg5_e1c875ab-aa59-4235-ac20-201d722fcc7f/egress-router-binary-copy/0.log"
Jul 20 23:10:42.376603 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:42.376580    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g7tg5_e1c875ab-aa59-4235-ac20-201d722fcc7f/cni-plugins/0.log"
Jul 20 23:10:42.394978 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:42.394957    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g7tg5_e1c875ab-aa59-4235-ac20-201d722fcc7f/bond-cni-plugin/0.log"
Jul 20 23:10:42.417458 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:42.417435    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g7tg5_e1c875ab-aa59-4235-ac20-201d722fcc7f/routeoverride-cni/0.log"
Jul 20 23:10:42.436391 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:42.436375    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g7tg5_e1c875ab-aa59-4235-ac20-201d722fcc7f/whereabouts-cni-bincopy/0.log"
Jul 20 23:10:42.455273 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:42.455251    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g7tg5_e1c875ab-aa59-4235-ac20-201d722fcc7f/whereabouts-cni/0.log"
Jul 20 23:10:42.546418 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:42.546392    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xsrgm_2c4d3266-7aaa-4d42-952c-f98e4e29dec4/kube-multus/0.log"
Jul 20 23:10:42.615295 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:42.615217    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-m5bvh_116caf25-c9e5-47da-a793-bf52c4aa1f5e/network-metrics-daemon/0.log"
Jul 20 23:10:42.633775 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:42.633753    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-m5bvh_116caf25-c9e5-47da-a793-bf52c4aa1f5e/kube-rbac-proxy/0.log"
Jul 20 23:10:42.814498 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:42.814455    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-85xd4_2b9d8d5a-7d89-4556-b963-ef432f0b749e/konnectivity-agent/0.log" limit=1024
Jul 20 23:10:43.687225 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:43.687195    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-controller/0.log"
Jul 20 23:10:43.702918 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:43.702889    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/0.log"
Jul 20 23:10:43.703046 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:43.702953    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovn-acl-logging/1.log"
Jul 20 23:10:43.723653 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:43.723630    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/kube-rbac-proxy-node/0.log"
Jul 20 23:10:43.743106 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:43.743081    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/kube-rbac-proxy-ovn-metrics/0.log"
Jul 20 23:10:43.760340 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:43.760323    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/northd/0.log"
Jul 20 23:10:43.778835 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:43.778818    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/nbdb/0.log"
Jul 20 23:10:43.797555 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:43.797535    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/sbdb/0.log"
Jul 20 23:10:43.847945 ip-10-0-129-16 kubenswrapper[2550]: I0720 23:10:43.847929    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h8985_87b10474-5ce4-419f-bf63-d9d4b32e7861/ovnkube-controller/0.log"