Jul 13 21:56:27.788732 ip-10-0-143-101 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 13 21:56:27.788741 ip-10-0-143-101 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 13 21:56:27.788747 ip-10-0-143-101 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 13 21:56:27.788962 ip-10-0-143-101 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 13 21:56:38.038489 ip-10-0-143-101 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 13 21:56:38.038506 ip-10-0-143-101 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot eb68f37e8443404e9c88f4b8eca700fe --
Jul 13 21:58:31.951814 ip-10-0-143-101 systemd[1]: Starting Kubernetes Kubelet...
Jul 13 21:58:32.423069 ip-10-0-143-101 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 21:58:32.423069 ip-10-0-143-101 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 13 21:58:32.423069 ip-10-0-143-101 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 21:58:32.423069 ip-10-0-143-101 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 13 21:58:32.423069 ip-10-0-143-101 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 21:58:32.424637 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.424496    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 13 21:58:32.432670 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432649    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:58:32.432670 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432668    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:58:32.432670 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432673    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432677    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432680    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432683    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432687    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432690    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432692    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432695    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432698    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432701    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432704    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432710    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432714    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432722    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432725    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432729    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432732    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432736    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432743    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:58:32.432776 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432746    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432750    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432752    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432756    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432759    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432762    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432765    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432768    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432771    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432773    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432776    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432779    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432783    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432786    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432789    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432792    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432796    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432799    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432801    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432804    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:58:32.433248 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432807    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432810    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432813    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432816    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432818    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432821    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432824    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432826    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432829    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432832    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432835    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432838    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432841    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432844    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432847    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432850    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432853    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432856    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432858    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432861    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:58:32.433793 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432864    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432867    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432870    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432872    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432875    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432877    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432880    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432883    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432886    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432889    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432891    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432894    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432897    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432899    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432903    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432905    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432908    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432911    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432913    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432916    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:58:32.434311 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432918    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432921    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432924    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432927    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432930    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432933    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432936    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432940    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432943    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432946    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432950    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432954    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432957    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432960    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432964    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432967    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432969    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.432972    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433729    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:58:32.434823 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433739    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433743    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433747    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433750    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433753    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433756    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433759    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433762    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433773    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433776    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433779    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433783    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433787    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433790    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433793    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433796    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433805    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433809    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433811    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433815    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:58:32.435282 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433818    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433821    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433824    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433826    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433829    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433832    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433835    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433838    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433840    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433843    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433846    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433848    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433851    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433853    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433856    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433859    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433861    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433864    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433867    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433869    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:58:32.435803 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433872    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433876    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433878    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433881    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433884    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433886    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433889    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433892    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433894    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433897    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433900    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433903    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433907    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433909    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433912    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433915    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433917    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433920    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433924    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433926    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:58:32.436309 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433929    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433932    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433934    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433937    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433940    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433942    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433945    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433948    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433951    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433954    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433957    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433959    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433962    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433965    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433968    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433971    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433974    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433977    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433980    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433982    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:58:32.436890 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433985    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433988    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433991    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433994    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433996    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.433999    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434002    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434004    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434008    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434011    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434014    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434017    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434019    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434022    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434025    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434028    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434031    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434035    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434434    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:58:32.437411 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434440    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434443    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434446    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434450    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434452    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434455    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434458    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434467    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434470    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434473    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434476    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434479    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434481    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434484    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434487    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434490    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434493    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434496    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434499    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434501    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:58:32.437942 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434504    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434507    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434510    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434512    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434515    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434517    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434521    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434524    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434526    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434529    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434531    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434536    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434542    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434545    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434548    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434551    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434554    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434557    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434560    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:58:32.438472 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434562    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434566    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434570    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434573    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434575    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434578    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434580    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434583    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434586    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434589    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434592    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434594    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434597    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434600    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434603    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434605    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434608    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434611    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434632    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434635    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434637    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:58:32.438970 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434640    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434643    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434646    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434648    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434653    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434656    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434658    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434661    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434664    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434668    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434672    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434675    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434678    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434681    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434684    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434687    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434689    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434692    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434695    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:58:32.439550 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434697    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434700    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434703    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434705    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434708    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434711    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434714    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434716    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434719    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434721    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434724    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434728    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434731    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434734    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434737    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434739    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434742    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434746    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.434748    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434829    2558 flags.go:64] FLAG: --address="0.0.0.0"
Jul 13 21:58:32.440056 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434840    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434851    2558 flags.go:64] FLAG: --anonymous-auth="true"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434871    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434878    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434882    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434886    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434890    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434895    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434898    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434902    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434905    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434910    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434914    2558 flags.go:64] FLAG: --cgroup-root=""
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434917    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434920    2558 flags.go:64] FLAG: --client-ca-file=""
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434923    2558 flags.go:64] FLAG: --cloud-provider="external"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434926    2558 flags.go:64] FLAG: --cluster-dns="[]"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434930    2558 flags.go:64] FLAG: --cluster-domain=""
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434933    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434936    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434939    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434943    2558 flags.go:64] FLAG: --container-log-max-files="5"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434946    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 13 21:58:32.440555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434950    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434953    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434956    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434959    2558 flags.go:64] FLAG: --contention-profiling="false"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434962    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434965    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434970    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434973    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434977    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434980    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434983    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434986    2558 flags.go:64] FLAG: --enable-load-reader="false"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434990    2558 flags.go:64] FLAG: --enable-server="true"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434993    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.434998    2558 flags.go:64] FLAG: --event-burst="100"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435001    2558 flags.go:64] FLAG: --event-qps="50"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435005    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435008    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435012    2558 flags.go:64] FLAG: --eviction-hard=""
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435016    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435020    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435023    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435026    2558 flags.go:64] FLAG: --eviction-soft=""
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435029    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435032    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 13 21:58:32.441225 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435035    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435038    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435041    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435044    2558 flags.go:64] FLAG: --fail-swap-on="true"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435047    2558 flags.go:64] FLAG: --feature-gates=""
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435051    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435054    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435057    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435060    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435063    2558 flags.go:64] FLAG: --healthz-port="10248"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435067    2558 flags.go:64] FLAG: --help="false"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435070    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435073    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435076    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435079    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435083    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435086    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435089    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435092    2558 flags.go:64] FLAG: --image-service-endpoint=""
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435095    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435098    2558 flags.go:64] FLAG: --kube-api-burst="100"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435101    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435104    2558 flags.go:64] FLAG: --kube-api-qps="50"
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435108    2558 flags.go:64] FLAG: --kube-reserved=""
Jul 13 21:58:32.441901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435110    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435114    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435117    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435122    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435125    2558 flags.go:64] FLAG: --lock-file=""
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435128    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435131    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435134    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435140    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435143    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435145    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435148    2558 flags.go:64] FLAG: --logging-format="text"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435152    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435155    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435158    2558 flags.go:64] FLAG: --manifest-url=""
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435161    2558 flags.go:64] FLAG: --manifest-url-header=""
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435166    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435169    2558 flags.go:64] FLAG: --max-open-files="1000000"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435173    2558 flags.go:64] FLAG: --max-pods="110"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435176    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435179    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435183    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435186    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435189    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435192    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 13 21:58:32.442527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435195    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435203    2558 flags.go:64] FLAG: --node-status-max-images="50"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435206    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435209    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435212    2558 flags.go:64] FLAG: --pod-cidr=""
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435215    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435221    2558 flags.go:64] FLAG: --pod-manifest-path=""
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435224    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435227    2558 flags.go:64] FLAG: --pods-per-core="0"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435230    2558 flags.go:64] FLAG: --port="10250"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435234    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435238    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0772dcbdacedb2897"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435241    2558 flags.go:64] FLAG: --qos-reserved=""
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435244    2558 flags.go:64] FLAG: --read-only-port="10255"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435247    2558 flags.go:64] FLAG: --register-node="true"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435250    2558 flags.go:64] FLAG: --register-with-taints=""
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435254    2558 flags.go:64] FLAG: --registry-burst="10"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435257    2558 flags.go:64] FLAG: --registry-qps="5"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435260    2558 flags.go:64] FLAG: --reserved-cpus=""
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435263    2558 flags.go:64] FLAG: --reserved-memory=""
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435266    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435269    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435272    2558 flags.go:64] FLAG: --rotate-certificates="false"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435275    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435278    2558 flags.go:64] FLAG: --runonce="false"
Jul 13 21:58:32.443174 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435281    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435285    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435288    2558 flags.go:64] FLAG: --seccomp-default="false"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435291    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435294    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435297    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435300    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435303    2558 flags.go:64] FLAG: --storage-driver-password="root"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435307    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435310    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435312    2558 flags.go:64] FLAG: --storage-driver-user="root"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435315    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435318    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435321    2558 flags.go:64] FLAG: --system-cgroups=""
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435324    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435330    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435333    2558 flags.go:64] FLAG: --tls-cert-file=""
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435336    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435363    2558 flags.go:64] FLAG: --tls-min-version=""
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435367    2558 flags.go:64] FLAG: --tls-private-key-file=""
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435370    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435373    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435376    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435379    2558 flags.go:64] FLAG: --v="2"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435383    2558 flags.go:64] FLAG: --version="false"
Jul 13 21:58:32.443808 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435387    2558 flags.go:64] FLAG: --vmodule=""
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435391    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.435394    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435526    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435530    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435533    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435536    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435539    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435541    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435544    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435548    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435551    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435554    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435557    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435559    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435562    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435564    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435567    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435570    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435573    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:58:32.444470 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435575    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435578    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435581    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435583    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435586    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435590    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435593    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435596    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435600    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435605    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435608    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435611    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435628    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435631    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435634    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435637    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435640    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435642    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435645    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435648    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:58:32.444982 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435651    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435654    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435658    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435661    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435663    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435666    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435669    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435672    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435674    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435677    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435680    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435683    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435686    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435689    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435691    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435694    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435697    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435700    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435703    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435706    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:58:32.445532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435708    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435711    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435713    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435716    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435719    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435721    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435724    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435727    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435729    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435732    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435735    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435737    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435740    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435743    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435746    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435749    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435752    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435755    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435758    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435760    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:58:32.446066 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435763    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435765    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435768    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435771    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435773    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435776    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435779    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435781    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435784    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435787    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435790    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435794    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435798    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435801    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435804    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435808    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435811    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435814    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435817    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:58:32.446662 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435820    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435823    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.435827    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.436588    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.444083    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.444107    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.444128    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.444134    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444200    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444205    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444209    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444212    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444215    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444218    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:58:32.447148 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444221    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444225    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444229    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444232    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444235    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444238    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444240    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444243    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444245    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444248    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444251    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444254    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444256    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444259    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444262    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444264    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444267    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444269    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444272    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444275    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:58:32.447509 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444277    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444280    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444283    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444292    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444296    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444299    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444301    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444304    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444307    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444309    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444312    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444316    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444320    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444323    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444326    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444329    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444332    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444335    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444337    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:58:32.448272 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444340    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444343    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444346    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444349    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444352    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444355    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444357    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444360    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444363    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444365    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444368    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444370    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444373    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444376    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444378    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444381    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444384    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444392    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444395    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444398    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:58:32.448988 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444401    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444404    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444407    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444409    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444412    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444421    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444424    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444427    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444430    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444432    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444435    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444438    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444442    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444445    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444449    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444452    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444455    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444457    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444460    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:58:32.449545 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444463    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444465    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444468    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444470    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444473    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444476    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444479    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444482    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444485    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444488    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444491    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444501    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444504    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444507    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444509    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:58:32.450166 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.444515    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444680    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444685    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444689    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444693    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444697    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444700    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444703    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444706    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444708    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444711    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444714    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444716    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444719    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444722    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444724    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444727    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444729    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444732    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444735    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:58:32.450574 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444738    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444741    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444743    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444746    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444749    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444751    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444754    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444757    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444759    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444769    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444773    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444775    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444778    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444780    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444783    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444786    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444788    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444791    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444794    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444796    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:58:32.451104 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444799    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444802    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444804    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444807    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444809    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444812    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444814    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444817    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444819    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444822    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444825    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444827    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444830    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444832    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444835    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444837    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444840    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444842    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444847    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444851    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:58:32.451602 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444853    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444857    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444867    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444870    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444873    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444876    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444878    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444881    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444884    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444887    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444890    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444892    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444895    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444897    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444900    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444902    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444905    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444908    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444910    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:58:32.452117 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444913    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444915    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444918    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444920    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444923    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444925    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444928    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444930    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444933    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444936    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444938    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444941    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444944    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444946    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444949    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444953    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444962    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444965    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444968    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444971    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:58:32.452710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:32.444974    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:58:32.453257 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.444978    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 21:58:32.453257 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.445145    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 13 21:58:32.453257 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.447433    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 13 21:58:32.453257 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.450027    2558 server.go:1019] "Starting client certificate rotation"
Jul 13 21:58:32.453257 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.450148    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 21:58:32.453257 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.451121    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 21:58:32.453257 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.453141    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 21:58:32.454422 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.454404    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 21:58:32.467458 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.467441    2558 log.go:25] "Validated CRI v1 runtime API"
Jul 13 21:58:32.470077 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.470059    2558 log.go:25] "Validated CRI v1 image API"
Jul 13 21:58:32.470981 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.470963    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 13 21:58:32.474175 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.474150    2558 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8449ee9c-0416-4dff-afbb-6f35d60ee2ab:/dev/nvme0n1p4 d735c78b-6d4d-4898-b2e5-792e2afaee7d:/dev/nvme0n1p3]
Jul 13 21:58:32.474241 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.474175    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 13 21:58:32.478685 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.478571    2558 manager.go:216] Machine: {Timestamp:2026-07-13 21:58:32.478088612 +0000 UTC m=+0.396065069 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3224626 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2673bc5164706717cba5322a3acdd2 SystemUUID:ec2673bc-5164-7067-17cb-a5322a3acdd2 BootID:eb68f37e-8443-404e-9c88-f4b8eca700fe Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f0:18:6f:b9:09 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f0:18:6f:b9:09 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:22:e9:c2:85:dd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 13 21:58:32.478685 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.478680    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 13 21:58:32.478783 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.478763    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 13 21:58:32.482552 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.482525    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 13 21:58:32.482743 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.482554    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-101.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 13 21:58:32.484405 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.484391    2558 topology_manager.go:138] "Creating topology manager with none policy"
Jul 13 21:58:32.484466 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.484411    2558 container_manager_linux.go:309] "Creating device plugin manager"
Jul 13 21:58:32.484466 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.484424    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 21:58:32.484466 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.484447    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 21:58:32.484571 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.484549    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 21:58:32.486042 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.486027    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 13 21:58:32.489752 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.489738    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 21:58:32.489901 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.489890    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 13 21:58:32.489962 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.489928    2558 kubelet.go:486] "Attempting to sync node with API server"
Jul 13 21:58:32.489962 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.489944    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 13 21:58:32.490060 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.489982    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 13 21:58:32.490060 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.489998    2558 kubelet.go:398] "Adding apiserver pod source"
Jul 13 21:58:32.490060 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.490011    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 13 21:58:32.490465 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.490448    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 21:58:32.490529 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.490475    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 21:58:32.493526 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.493508    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 13 21:58:32.497620 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.497601    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 13 21:58:32.497699 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.497653    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 13 21:58:32.497835 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.497824    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 13 21:58:32.497867 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.497840    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 13 21:58:32.497867 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.497847    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 13 21:58:32.497867 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.497853    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 13 21:58:32.497867 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.497861    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 13 21:58:32.497867 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.497867    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 13 21:58:32.498580 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.498571    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 13 21:58:32.498580 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.498580    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 13 21:58:32.498669 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.498587    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 13 21:58:32.498669 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.498592    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 13 21:58:32.498669 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.498601    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 13 21:58:32.498669 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.498609    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 13 21:58:32.499925 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.499914    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 13 21:58:32.499925 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.499925    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 13 21:58:32.501783 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.501763    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-101.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 13 21:58:32.501868 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.501779    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 13 21:58:32.501868 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.501829    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 13 21:58:32.503129 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.503115    2558 server.go:1268] "Started kubelet"
Jul 13 21:58:32.503240 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.503193    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 13 21:58:32.503306 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.503263    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 13 21:58:32.503306 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.503196    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 13 21:58:32.503933 ip-10-0-143-101 systemd[1]: Started Kubernetes Kubelet.
Jul 13 21:58:32.504686 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.504585    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 13 21:58:32.505478 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.505464    2558 server.go:310] "Adding debug handlers to kubelet server"
Jul 13 21:58:32.511793 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.511773    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 13 21:58:32.512148 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.512130    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 13 21:58:32.512242 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.512172    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 13 21:58:32.512242 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.512222    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:32.513065 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.513045    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mfpdr"
Jul 13 21:58:32.513157 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.513109    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 13 21:58:32.513157 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.513111    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 13 21:58:32.513157 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.513121    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 13 21:58:32.513157 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.513148    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 13 21:58:32.513344 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.513161    2558 factory.go:55] Registering systemd factory
Jul 13 21:58:32.513344 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.513172    2558 factory.go:223] Registration of the systemd container factory successfully
Jul 13 21:58:32.513344 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.513318    2558 reconstruct.go:97] "Volume reconstruction finished"
Jul 13 21:58:32.513344 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.513327    2558 reconciler.go:29] "Reconciler: start to sync state"
Jul 13 21:58:32.513550 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.513425    2558 factory.go:153] Registering CRI-O factory
Jul 13 21:58:32.513550 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.513440    2558 factory.go:223] Registration of the crio container factory successfully
Jul 13 21:58:32.513550 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.513464    2558 factory.go:103] Registering Raw factory
Jul 13 21:58:32.513550 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.513504    2558 manager.go:1196] Started watching for new ooms in manager
Jul 13 21:58:32.514100 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.514069    2558 manager.go:349] Starting recovery of all containers
Jul 13 21:58:32.515695 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.513859    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-101.ec2.internal.18c1f8474b2dd81d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-101.ec2.internal,UID:ip-10-0-143-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-101.ec2.internal,},FirstTimestamp:2026-07-13 21:58:32.503089181 +0000 UTC m=+0.421065639,LastTimestamp:2026-07-13 21:58:32.503089181 +0000 UTC m=+0.421065639,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-101.ec2.internal,}"
Jul 13 21:58:32.518705 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.518675    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 13 21:58:32.518799 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.518689    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 13 21:58:32.522334 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.522311    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mfpdr"
Jul 13 21:58:32.526498 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.526479    2558 manager.go:354] Recovery completed
Jul 13 21:58:32.529093 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.529066    2558 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 13 21:58:32.532422 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.532409    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:58:32.534511 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.534492    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:58:32.534588 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.534528    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:58:32.534588 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.534543    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:58:32.535149 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.535133    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 13 21:58:32.535149 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.535147    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 13 21:58:32.535226 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.535162    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 21:58:32.535226 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.535179    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:58:32.536002 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.535980    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:58:32.536111 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.536011    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:58:32.536111 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.536026    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:58:32.536227 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.536124    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-101.ec2.internal.18c1f8474d0d4e17  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-101.ec2.internal,UID:ip-10-0-143-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-101.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-101.ec2.internal,},FirstTimestamp:2026-07-13 21:58:32.534511127 +0000 UTC m=+0.452487590,LastTimestamp:2026-07-13 21:58:32.534511127 +0000 UTC m=+0.452487590,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-101.ec2.internal,}"
Jul 13 21:58:32.536482 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.536469    2558 policy_none.go:49] "None policy: Start"
Jul 13 21:58:32.536527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.536485    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 13 21:58:32.536527 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.536496    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 13 21:58:32.537818 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.537709    2558 policy_none.go:47] "Start"
Jul 13 21:58:32.577198 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.577182    2558 manager.go:337] "Starting Device Plugin manager"
Jul 13 21:58:32.577270 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.577212    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 13 21:58:32.577270 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.577224    2558 server.go:87] "Starting device plugin registration server"
Jul 13 21:58:32.577477 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.577460    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 13 21:58:32.577566 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.577475    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 13 21:58:32.577668 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.577574    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 13 21:58:32.577722 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.577681    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 13 21:58:32.577722 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.577692    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 13 21:58:32.578232 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.578213    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 13 21:58:32.578302 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.578267    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:32.658279 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.658226    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 13 21:58:32.660454 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.659507    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 13 21:58:32.660454 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.659536    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 13 21:58:32.660454 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.659556    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 13 21:58:32.660454 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.659587    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 13 21:58:32.661946 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.661930    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:58:32.678665 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.678649    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:58:32.679471 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.679446    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:58:32.679577 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.679481    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:58:32.679577 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.679497    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:58:32.679577 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.679526    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.689448 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.689433    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.689519 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.689453    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-101.ec2.internal\": node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:32.721764 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.721741    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:32.760657 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.760634    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-101.ec2.internal"]
Jul 13 21:58:32.760720 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.760695    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:58:32.761478 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.761459    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:58:32.761602 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.761483    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:58:32.761602 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.761493    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:58:32.766415 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.766402    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:58:32.766594 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.766578    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.766684 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.766615    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:58:32.767136 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.767109    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:58:32.767136 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.767133    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:58:32.767259 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.767143    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:58:32.767259 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.767132    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:58:32.767259 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.767217    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:58:32.767259 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.767228    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:58:32.769393 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.769377    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.769450 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.769403    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:58:32.770107 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.770091    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:58:32.770190 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.770126    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:58:32.770190 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.770140    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:58:32.789264 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.789242    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-101.ec2.internal\" not found" node="ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.793750 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.793736    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-101.ec2.internal\" not found" node="ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.815376 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.815358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/40671ae6695f47aabb33292598c63a42-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal\" (UID: \"40671ae6695f47aabb33292598c63a42\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.815441 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.815383    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40671ae6695f47aabb33292598c63a42-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal\" (UID: \"40671ae6695f47aabb33292598c63a42\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.815441 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.815404    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b7f01a0953299e98dac4176c400f9a5-config\") pod \"kube-apiserver-proxy-ip-10-0-143-101.ec2.internal\" (UID: \"3b7f01a0953299e98dac4176c400f9a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.822812 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.822794    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:32.916209 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.916153    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40671ae6695f47aabb33292598c63a42-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal\" (UID: \"40671ae6695f47aabb33292598c63a42\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.916209 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.916185    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b7f01a0953299e98dac4176c400f9a5-config\") pod \"kube-apiserver-proxy-ip-10-0-143-101.ec2.internal\" (UID: \"3b7f01a0953299e98dac4176c400f9a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.916339 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.916213    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/40671ae6695f47aabb33292598c63a42-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal\" (UID: \"40671ae6695f47aabb33292598c63a42\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.916339 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.916248    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40671ae6695f47aabb33292598c63a42-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal\" (UID: \"40671ae6695f47aabb33292598c63a42\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.916339 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.916264    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b7f01a0953299e98dac4176c400f9a5-config\") pod \"kube-apiserver-proxy-ip-10-0-143-101.ec2.internal\" (UID: \"3b7f01a0953299e98dac4176c400f9a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.916339 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:32.916249    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/40671ae6695f47aabb33292598c63a42-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal\" (UID: \"40671ae6695f47aabb33292598c63a42\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:32.923875 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:32.923856    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:33.024057 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:33.024037    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:33.096760 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.096736    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:33.097917 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.097896    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:33.124330 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:33.124302    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:33.224464 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:33.224384    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:33.317518 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.317497    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:58:33.325433 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:33.325415    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:33.426273 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:33.426250    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:33.449076 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.449055    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 13 21:58:33.449205 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.449188    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 21:58:33.449245 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.449215    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 21:58:33.474612 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.474557    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:58:33.527168 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:33.527141    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:33.527286 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.527174    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 21:53:32 +0000 UTC" deadline="2028-04-17 23:42:05.662276706 +0000 UTC"
Jul 13 21:58:33.527286 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.527198    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15457h43m32.135081433s"
Jul 13 21:58:33.627953 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:33.627926    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:33.639977 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:33.639935    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40671ae6695f47aabb33292598c63a42.slice/crio-f5d5161f84f38b3dd1847d0c01af0622703a206283525adeda19c441c58963b6 WatchSource:0}: Error finding container f5d5161f84f38b3dd1847d0c01af0622703a206283525adeda19c441c58963b6: Status 404 returned error can't find the container with id f5d5161f84f38b3dd1847d0c01af0622703a206283525adeda19c441c58963b6
Jul 13 21:58:33.645783 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.645765    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:58:33.647692 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:33.647662    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b7f01a0953299e98dac4176c400f9a5.slice/crio-241d96e3a2a47911f593b359a36746b9dca0ccc22326a853ed6ba252a8880686 WatchSource:0}: Error finding container 241d96e3a2a47911f593b359a36746b9dca0ccc22326a853ed6ba252a8880686: Status 404 returned error can't find the container with id 241d96e3a2a47911f593b359a36746b9dca0ccc22326a853ed6ba252a8880686
Jul 13 21:58:33.663497 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.663453    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-101.ec2.internal" event={"ID":"3b7f01a0953299e98dac4176c400f9a5","Type":"ContainerStarted","Data":"241d96e3a2a47911f593b359a36746b9dca0ccc22326a853ed6ba252a8880686"}
Jul 13 21:58:33.664459 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.664430    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal" event={"ID":"40671ae6695f47aabb33292598c63a42","Type":"ContainerStarted","Data":"f5d5161f84f38b3dd1847d0c01af0622703a206283525adeda19c441c58963b6"}
Jul 13 21:58:33.728148 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:33.728090    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:33.828158 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:33.828133    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-101.ec2.internal\" not found"
Jul 13 21:58:33.849347 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.849324    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:58:33.916099 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.916073    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:33.930813 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.930794    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 21:58:33.932221 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.932208    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-101.ec2.internal"
Jul 13 21:58:33.939593 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:33.939576    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 21:58:34.491162 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.490859    2558 apiserver.go:52] "Watching apiserver"
Jul 13 21:58:34.513457 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.513432    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 13 21:58:34.515278 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.515235    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 13 21:58:34.515791 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.515765    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-7n4kx","kube-system/kube-apiserver-proxy-ip-10-0-143-101.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26","openshift-cluster-node-tuning-operator/tuned-zkwrx","openshift-dns/node-resolver-p75ql","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal","openshift-multus/multus-vht6c","openshift-multus/network-metrics-daemon-bx7hm","kube-system/konnectivity-agent-k5gj5","openshift-image-registry/node-ca-w98nt","openshift-multus/multus-additional-cni-plugins-czcbx","openshift-network-diagnostics/network-check-target-csbjt","openshift-network-operator/iptables-alerter-q4kp4","openshift-ovn-kubernetes/ovnkube-node-x6k99"]
Jul 13 21:58:34.519015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.518992    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.521651 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.521627    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8lhts\""
Jul 13 21:58:34.522035 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.522013    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 13 21:58:34.522175 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.522157    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q4kp4"
Jul 13 21:58:34.522296 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.522260    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 13 21:58:34.522296 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.522269    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 13 21:58:34.522417 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.522358    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 13 21:58:34.524161 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.524142    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.524784 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.524761    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-k8kxb\""
Jul 13 21:58:34.524886 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.524788    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 13 21:58:34.525119 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.525098    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 21:58:34.525198 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.525097    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 13 21:58:34.525198 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.525157    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:58:34.526648 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.526503    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 13 21:58:34.527025 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.526991    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 13 21:58:34.527217 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.527198    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-25qlw\""
Jul 13 21:58:34.527398 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.527384    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 13 21:58:34.529006 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.528986    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.531204 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.531182    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 13 21:58:34.531442 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.531419    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:58:34.531559 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.531423    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-c5vbl\""
Jul 13 21:58:34.531891 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.531863    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p75ql"
Jul 13 21:58:34.534400 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.534044    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nldqr\""
Jul 13 21:58:34.534498 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.534319    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 13 21:58:34.534648 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.534376    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 13 21:58:34.535975 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.535956    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.538277 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.538261    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 13 21:58:34.538472 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.538457    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 13 21:58:34.538650 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.538629    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 13 21:58:34.538726 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.538653    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 13 21:58:34.539095 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.539076    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xvwcq\""
Jul 13 21:58:34.539178 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.539117    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k5gj5"
Jul 13 21:58:34.539710 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.539691    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 13 21:58:34.539908 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.539890    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 13 21:58:34.542074 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.542056    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:34.542187 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:34.542161    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:58:34.543986 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.543966    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:34.544085 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:34.544032    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:58:34.545796 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.545777    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 13 21:58:34.546026 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.546002    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 13 21:58:34.546096 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.546031    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qcrdl\""
Jul 13 21:58:34.547149 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.547130    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w98nt"
Jul 13 21:58:34.550896 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.550847    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.551811 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.551779    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 13 21:58:34.553308 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.552023    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bd99z\""
Jul 13 21:58:34.553308 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.552273    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 13 21:58:34.553308 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.552291    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 13 21:58:34.554821 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.554664    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:34.554821 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:34.554747    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:58:34.557119 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.557099    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 13 21:58:34.557364 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.557344    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ngg2p\""
Jul 13 21:58:34.557442 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.557389    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 13 21:58:34.615371 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.615351    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 13 21:58:34.623939 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.623911    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-run-systemd\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.624073 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.623953    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-var-lib-openvswitch\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.624073 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.623983    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-run-openvswitch\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.624073 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624023    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dqwx\" (UniqueName: \"kubernetes.io/projected/b485735e-e460-453b-8503-c034e753bf78-kube-api-access-5dqwx\") pod \"network-metrics-daemon-bx7hm\" (UID: \"b485735e-e460-453b-8503-c034e753bf78\") " pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:34.624073 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624049    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-var-lib-kubelet\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.624073 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624073    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-sysctl-d\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.624352 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624098    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-slash\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.624352 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624123    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-run-ovn\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.624352 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624150    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.624352 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624184    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h7vj\" (UniqueName: \"kubernetes.io/projected/805523c8-888a-45cb-8291-6ee665e8eeb1-kube-api-access-7h7vj\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.624352 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624208    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-device-dir\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.624352 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2948a7e0-5979-4656-9e00-c2cdde279ab2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.624352 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-run\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.624352 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624283    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-run-netns\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.624352 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624300    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-node-log\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.624352 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624318    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/68f721b5-75ba-441c-a277-cb34adc92f53-env-overrides\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.624352 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624344    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2948a7e0-5979-4656-9e00-c2cdde279ab2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624372    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn5zx\" (UniqueName: \"kubernetes.io/projected/2948a7e0-5979-4656-9e00-c2cdde279ab2-kube-api-access-gn5zx\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624392    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-var-lib-kubelet\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624421    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/81356c52-66f3-4252-bb9a-d62a47236627-etc-tuned\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624444    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/68f721b5-75ba-441c-a277-cb34adc92f53-ovnkube-script-lib\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624472    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/191c77b5-4225-4b5d-9402-ccdc1765dde8-dbus\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624495    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-multus-conf-dir\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624520    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2948a7e0-5979-4656-9e00-c2cdde279ab2-cni-binary-copy\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624544    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-modprobe-d\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624571    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-sysconfig\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624597    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624642    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1820ff41-2d99-4666-b910-90b3110b7e32-hosts-file\") pod \"node-resolver-p75ql\" (UID: \"1820ff41-2d99-4666-b910-90b3110b7e32\") " pod="openshift-dns/node-resolver-p75ql"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624683    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qzvc\" (UniqueName: \"kubernetes.io/projected/1820ff41-2d99-4666-b910-90b3110b7e32-kube-api-access-6qzvc\") pod \"node-resolver-p75ql\" (UID: \"1820ff41-2d99-4666-b910-90b3110b7e32\") " pod="openshift-dns/node-resolver-p75ql"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624708    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-kubelet\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624732    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/191c77b5-4225-4b5d-9402-ccdc1765dde8-kubelet-config\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624759    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-system-cni-dir\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.625408 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624786    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-var-lib-cni-bin\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624818    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-kubernetes\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624876    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/68f721b5-75ba-441c-a277-cb34adc92f53-ovnkube-config\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624943    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzvk8\" (UniqueName: \"kubernetes.io/projected/113d2c9e-31c0-44f6-b14f-a1e363928d87-kube-api-access-qzvk8\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.624987    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drljw\" (UniqueName: \"kubernetes.io/projected/a27ef85e-b4e2-4778-855e-7a05bfeb4a00-kube-api-access-drljw\") pod \"node-ca-w98nt\" (UID: \"a27ef85e-b4e2-4778-855e-7a05bfeb4a00\") " pod="openshift-image-registry/node-ca-w98nt"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625014    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-sysctl-conf\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625037    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-host\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625059    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81356c52-66f3-4252-bb9a-d62a47236627-tmp\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625083    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xknk\" (UniqueName: \"kubernetes.io/projected/68f721b5-75ba-441c-a277-cb34adc92f53-kube-api-access-7xknk\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625110    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-registration-dir\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625133    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a27ef85e-b4e2-4778-855e-7a05bfeb4a00-serviceca\") pod \"node-ca-w98nt\" (UID: \"a27ef85e-b4e2-4778-855e-7a05bfeb4a00\") " pod="openshift-image-registry/node-ca-w98nt"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625156    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm58v\" (UniqueName: \"kubernetes.io/projected/81356c52-66f3-4252-bb9a-d62a47236627-kube-api-access-wm58v\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625189    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-hostroot\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625213    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ccsd\" (UniqueName: \"kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd\") pod \"network-check-target-csbjt\" (UID: \"2ad8149f-fe74-4aa2-8f43-bef582499a31\") " pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625238    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99237eaa-d7e0-42b7-892c-d8ec6fe1239f-iptables-alerter-script\") pod \"iptables-alerter-q4kp4\" (UID: \"99237eaa-d7e0-42b7-892c-d8ec6fe1239f\") " pod="openshift-network-operator/iptables-alerter-q4kp4"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625266    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-systemd-units\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.626231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625291    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-multus-cni-dir\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625318    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-run-netns\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625342    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-log-socket\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625366    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a27ef85e-b4e2-4778-855e-7a05bfeb4a00-host\") pod \"node-ca-w98nt\" (UID: \"a27ef85e-b4e2-4778-855e-7a05bfeb4a00\") " pod="openshift-image-registry/node-ca-w98nt"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625415    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625440    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-run-multus-certs\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625465    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-socket-dir\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625487    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-etc-selinux\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625510    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99237eaa-d7e0-42b7-892c-d8ec6fe1239f-host-slash\") pod \"iptables-alerter-q4kp4\" (UID: \"99237eaa-d7e0-42b7-892c-d8ec6fe1239f\") " pod="openshift-network-operator/iptables-alerter-q4kp4"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625536    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-etc-kubernetes\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625559    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/02ee941b-f7e8-49a2-ad16-bf6c1da015ec-agent-certs\") pod \"konnectivity-agent-k5gj5\" (UID: \"02ee941b-f7e8-49a2-ad16-bf6c1da015ec\") " pod="kube-system/konnectivity-agent-k5gj5"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625584    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-etc-openvswitch\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625718    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-run-ovn-kubernetes\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625758    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/68f721b5-75ba-441c-a277-cb34adc92f53-ovn-node-metrics-cert\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625798    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs\") pod \"network-metrics-daemon-bx7hm\" (UID: \"b485735e-e460-453b-8503-c034e753bf78\") " pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625834    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-cnibin\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625868    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/805523c8-888a-45cb-8291-6ee665e8eeb1-cni-binary-copy\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.627015 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625916    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-os-release\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625944    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-multus-socket-dir-parent\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625970    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/02ee941b-f7e8-49a2-ad16-bf6c1da015ec-konnectivity-ca\") pod \"konnectivity-agent-k5gj5\" (UID: \"02ee941b-f7e8-49a2-ad16-bf6c1da015ec\") " pod="kube-system/konnectivity-agent-k5gj5"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.625995    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2948a7e0-5979-4656-9e00-c2cdde279ab2-system-cni-dir\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626058    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2948a7e0-5979-4656-9e00-c2cdde279ab2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626102    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4ld9\" (UniqueName: \"kubernetes.io/projected/99237eaa-d7e0-42b7-892c-d8ec6fe1239f-kube-api-access-k4ld9\") pod \"iptables-alerter-q4kp4\" (UID: \"99237eaa-d7e0-42b7-892c-d8ec6fe1239f\") " pod="openshift-network-operator/iptables-alerter-q4kp4"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626139    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-systemd\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626166    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-cni-bin\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626198    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-cni-netd\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626241    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/805523c8-888a-45cb-8291-6ee665e8eeb1-multus-daemon-config\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626267    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-lib-modules\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626308    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-sys-fs\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626342    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2948a7e0-5979-4656-9e00-c2cdde279ab2-cnibin\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626367    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2948a7e0-5979-4656-9e00-c2cdde279ab2-os-release\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626390    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-sys\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626413    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-run-k8s-cni-cncf-io\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.627942 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626437    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-var-lib-cni-multus\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.628752 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.626465    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1820ff41-2d99-4666-b910-90b3110b7e32-tmp-dir\") pod \"node-resolver-p75ql\" (UID: \"1820ff41-2d99-4666-b910-90b3110b7e32\") " pod="openshift-dns/node-resolver-p75ql"
Jul 13 21:58:34.638578 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.638528    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:58:34.661921 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.661896    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6x6v5"
Jul 13 21:58:34.670502 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.670476    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6x6v5"
Jul 13 21:58:34.727646 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.727616    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/68f721b5-75ba-441c-a277-cb34adc92f53-env-overrides\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.727787 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.727658    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2948a7e0-5979-4656-9e00-c2cdde279ab2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.727787 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.727684    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn5zx\" (UniqueName: \"kubernetes.io/projected/2948a7e0-5979-4656-9e00-c2cdde279ab2-kube-api-access-gn5zx\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.728103 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728077    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-var-lib-kubelet\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.728202 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728118    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/81356c52-66f3-4252-bb9a-d62a47236627-etc-tuned\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.728202 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728142    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/68f721b5-75ba-441c-a277-cb34adc92f53-ovnkube-script-lib\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.728202 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728166    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/191c77b5-4225-4b5d-9402-ccdc1765dde8-dbus\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:34.728202 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728189    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-multus-conf-dir\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.728398 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728212    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2948a7e0-5979-4656-9e00-c2cdde279ab2-cni-binary-copy\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.728398 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728236    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-modprobe-d\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.728398 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728262    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-sysconfig\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.728398 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728288    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.728398 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728318    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/191c77b5-4225-4b5d-9402-ccdc1765dde8-dbus\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:34.728398 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728312    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1820ff41-2d99-4666-b910-90b3110b7e32-hosts-file\") pod \"node-resolver-p75ql\" (UID: \"1820ff41-2d99-4666-b910-90b3110b7e32\") " pod="openshift-dns/node-resolver-p75ql"
Jul 13 21:58:34.728398 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728355    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qzvc\" (UniqueName: \"kubernetes.io/projected/1820ff41-2d99-4666-b910-90b3110b7e32-kube-api-access-6qzvc\") pod \"node-resolver-p75ql\" (UID: \"1820ff41-2d99-4666-b910-90b3110b7e32\") " pod="openshift-dns/node-resolver-p75ql"
Jul 13 21:58:34.728398 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728238    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2948a7e0-5979-4656-9e00-c2cdde279ab2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.728855 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728445    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1820ff41-2d99-4666-b910-90b3110b7e32-hosts-file\") pod \"node-resolver-p75ql\" (UID: \"1820ff41-2d99-4666-b910-90b3110b7e32\") " pod="openshift-dns/node-resolver-p75ql"
Jul 13 21:58:34.728855 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728473    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-multus-conf-dir\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.728855 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728473    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-var-lib-kubelet\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.728855 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728738    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/68f721b5-75ba-441c-a277-cb34adc92f53-env-overrides\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.728855 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728743    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 13 21:58:34.729108 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728876    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-modprobe-d\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.729108 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.728923    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-sysconfig\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.729108 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729011    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2948a7e0-5979-4656-9e00-c2cdde279ab2-cni-binary-copy\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.729108 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729048    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.729108 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729065    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-kubelet\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.729364 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729132    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/191c77b5-4225-4b5d-9402-ccdc1765dde8-kubelet-config\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:34.729364 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729171    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-kubelet\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.729364 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729234    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-system-cni-dir\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.729364 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729266    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-var-lib-cni-bin\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.729364 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729280    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-system-cni-dir\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.729364 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729290    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/191c77b5-4225-4b5d-9402-ccdc1765dde8-kubelet-config\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:34.729364 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729334    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-kubernetes\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.729666 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729379    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-var-lib-cni-bin\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.729666 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729413    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-kubernetes\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.729666 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729463    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/68f721b5-75ba-441c-a277-cb34adc92f53-ovnkube-config\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.729666 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729528    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzvk8\" (UniqueName: \"kubernetes.io/projected/113d2c9e-31c0-44f6-b14f-a1e363928d87-kube-api-access-qzvk8\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.729666 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729547    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/68f721b5-75ba-441c-a277-cb34adc92f53-ovnkube-script-lib\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.729666 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729610    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drljw\" (UniqueName: \"kubernetes.io/projected/a27ef85e-b4e2-4778-855e-7a05bfeb4a00-kube-api-access-drljw\") pod \"node-ca-w98nt\" (UID: \"a27ef85e-b4e2-4778-855e-7a05bfeb4a00\") " pod="openshift-image-registry/node-ca-w98nt"
Jul 13 21:58:34.729931 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729734    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-sysctl-conf\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.729931 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729842    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-host\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.729931 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729882    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/68f721b5-75ba-441c-a277-cb34adc92f53-ovnkube-config\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.729931 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729884    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81356c52-66f3-4252-bb9a-d62a47236627-tmp\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.730145 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729947    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xknk\" (UniqueName: \"kubernetes.io/projected/68f721b5-75ba-441c-a277-cb34adc92f53-kube-api-access-7xknk\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.730145 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729990    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-host\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.730145 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730001    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-sysctl-conf\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.730145 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.729992    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-registration-dir\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.730145 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730034    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a27ef85e-b4e2-4778-855e-7a05bfeb4a00-serviceca\") pod \"node-ca-w98nt\" (UID: \"a27ef85e-b4e2-4778-855e-7a05bfeb4a00\") " pod="openshift-image-registry/node-ca-w98nt"
Jul 13 21:58:34.730145 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730063    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm58v\" (UniqueName: \"kubernetes.io/projected/81356c52-66f3-4252-bb9a-d62a47236627-kube-api-access-wm58v\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.730145 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730089    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-hostroot\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.730145 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730111    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccsd\" (UniqueName: \"kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd\") pod \"network-check-target-csbjt\" (UID: \"2ad8149f-fe74-4aa2-8f43-bef582499a31\") " pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:34.730145 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730123    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-registration-dir\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.730145 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730135    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99237eaa-d7e0-42b7-892c-d8ec6fe1239f-iptables-alerter-script\") pod \"iptables-alerter-q4kp4\" (UID: \"99237eaa-d7e0-42b7-892c-d8ec6fe1239f\") " pod="openshift-network-operator/iptables-alerter-q4kp4"
Jul 13 21:58:34.730464 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730178    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-systemd-units\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.730464 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730206    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-multus-cni-dir\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.730464 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730208    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-hostroot\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.730464 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730253    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-run-netns\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.730464 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730278    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-multus-cni-dir\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.730464 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730278    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-log-socket\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.730464 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730316    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-log-socket\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.730464 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730317    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a27ef85e-b4e2-4778-855e-7a05bfeb4a00-host\") pod \"node-ca-w98nt\" (UID: \"a27ef85e-b4e2-4778-855e-7a05bfeb4a00\") " pod="openshift-image-registry/node-ca-w98nt"
Jul 13 21:58:34.730464 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730348    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a27ef85e-b4e2-4778-855e-7a05bfeb4a00-host\") pod \"node-ca-w98nt\" (UID: \"a27ef85e-b4e2-4778-855e-7a05bfeb4a00\") " pod="openshift-image-registry/node-ca-w98nt"
Jul 13 21:58:34.730464 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730356    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:34.730464 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730406    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-run-multus-certs\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.730464 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730433    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-socket-dir\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.730464 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730459    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-etc-selinux\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730484    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99237eaa-d7e0-42b7-892c-d8ec6fe1239f-host-slash\") pod \"iptables-alerter-q4kp4\" (UID: \"99237eaa-d7e0-42b7-892c-d8ec6fe1239f\") " pod="openshift-network-operator/iptables-alerter-q4kp4"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730508    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-etc-kubernetes\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730552    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/02ee941b-f7e8-49a2-ad16-bf6c1da015ec-agent-certs\") pod \"konnectivity-agent-k5gj5\" (UID: \"02ee941b-f7e8-49a2-ad16-bf6c1da015ec\") " pod="kube-system/konnectivity-agent-k5gj5"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730556    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-run-multus-certs\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730512    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a27ef85e-b4e2-4778-855e-7a05bfeb4a00-serviceca\") pod \"node-ca-w98nt\" (UID: \"a27ef85e-b4e2-4778-855e-7a05bfeb4a00\") " pod="openshift-image-registry/node-ca-w98nt"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730645    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-etc-kubernetes\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730684    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-etc-openvswitch\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730697    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99237eaa-d7e0-42b7-892c-d8ec6fe1239f-iptables-alerter-script\") pod \"iptables-alerter-q4kp4\" (UID: \"99237eaa-d7e0-42b7-892c-d8ec6fe1239f\") " pod="openshift-network-operator/iptables-alerter-q4kp4"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730688    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-etc-selinux\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730713    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-run-ovn-kubernetes\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730735    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-etc-openvswitch\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730779    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-run-ovn-kubernetes\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730738    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/68f721b5-75ba-441c-a277-cb34adc92f53-ovn-node-metrics-cert\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730820    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs\") pod \"network-metrics-daemon-bx7hm\" (UID: \"b485735e-e460-453b-8503-c034e753bf78\") " pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730823    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-run-netns\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730846    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-cnibin\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730871    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/805523c8-888a-45cb-8291-6ee665e8eeb1-cni-binary-copy\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.730900 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:34.730897    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730951    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-os-release\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:34.730985    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs podName:b485735e-e460-453b-8503-c034e753bf78 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:35.230945668 +0000 UTC m=+3.148922130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs") pod "network-metrics-daemon-bx7hm" (UID: "b485735e-e460-453b-8503-c034e753bf78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.730898    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-os-release\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731027    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-multus-socket-dir-parent\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731043    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-cnibin\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731055    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/02ee941b-f7e8-49a2-ad16-bf6c1da015ec-konnectivity-ca\") pod \"konnectivity-agent-k5gj5\" (UID: \"02ee941b-f7e8-49a2-ad16-bf6c1da015ec\") " pod="kube-system/konnectivity-agent-k5gj5"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731094    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2948a7e0-5979-4656-9e00-c2cdde279ab2-system-cni-dir\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731118    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2948a7e0-5979-4656-9e00-c2cdde279ab2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731144    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4ld9\" (UniqueName: \"kubernetes.io/projected/99237eaa-d7e0-42b7-892c-d8ec6fe1239f-kube-api-access-k4ld9\") pod \"iptables-alerter-q4kp4\" (UID: \"99237eaa-d7e0-42b7-892c-d8ec6fe1239f\") " pod="openshift-network-operator/iptables-alerter-q4kp4"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731169    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-systemd\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731186    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-cni-bin\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731202    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-cni-netd\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731212    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99237eaa-d7e0-42b7-892c-d8ec6fe1239f-host-slash\") pod \"iptables-alerter-q4kp4\" (UID: \"99237eaa-d7e0-42b7-892c-d8ec6fe1239f\") " pod="openshift-network-operator/iptables-alerter-q4kp4"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731221    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/805523c8-888a-45cb-8291-6ee665e8eeb1-multus-daemon-config\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731253    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-lib-modules\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731279    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-sys-fs\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.731856 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731307    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2948a7e0-5979-4656-9e00-c2cdde279ab2-cnibin\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731332    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2948a7e0-5979-4656-9e00-c2cdde279ab2-os-release\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731354    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-sys\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731379    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-run-k8s-cni-cncf-io\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731405    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-var-lib-cni-multus\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731429    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1820ff41-2d99-4666-b910-90b3110b7e32-tmp-dir\") pod \"node-resolver-p75ql\" (UID: \"1820ff41-2d99-4666-b910-90b3110b7e32\") " pod="openshift-dns/node-resolver-p75ql"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731455    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-run-systemd\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731482    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-var-lib-openvswitch\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731507    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-run-openvswitch\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731535    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dqwx\" (UniqueName: \"kubernetes.io/projected/b485735e-e460-453b-8503-c034e753bf78-kube-api-access-5dqwx\") pod \"network-metrics-daemon-bx7hm\" (UID: \"b485735e-e460-453b-8503-c034e753bf78\") " pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731562    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-var-lib-kubelet\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731585    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-sysctl-d\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731611    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-slash\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731639    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/805523c8-888a-45cb-8291-6ee665e8eeb1-multus-daemon-config\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731658    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-run-ovn\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731683    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731703    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-multus-socket-dir-parent\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.732630 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731711    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h7vj\" (UniqueName: \"kubernetes.io/projected/805523c8-888a-45cb-8291-6ee665e8eeb1-kube-api-access-7h7vj\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731732    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-run-openvswitch\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731782    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-var-lib-cni-multus\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731815    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-var-lib-kubelet\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731837    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2948a7e0-5979-4656-9e00-c2cdde279ab2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731825    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-sys-fs\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731881    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-run-systemd\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731903    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-slash\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731922    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2948a7e0-5979-4656-9e00-c2cdde279ab2-system-cni-dir\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731955    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-run-ovn\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.731988    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-var-lib-openvswitch\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732007    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732054    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-device-dir\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732086    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2948a7e0-5979-4656-9e00-c2cdde279ab2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732107    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-systemd\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732119    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-run\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732140    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-etc-sysctl-d\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.733524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732152    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-run-netns\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732183    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-node-log\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732186    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2948a7e0-5979-4656-9e00-c2cdde279ab2-os-release\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732083    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-socket-dir\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732224    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/805523c8-888a-45cb-8291-6ee665e8eeb1-host-run-k8s-cni-cncf-io\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732121    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-systemd-units\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732250    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-cni-bin\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732279    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-sys\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732332    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-run\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732398    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/113d2c9e-31c0-44f6-b14f-a1e363928d87-device-dir\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732405    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81356c52-66f3-4252-bb9a-d62a47236627-lib-modules\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732471    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-cni-netd\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732511    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1820ff41-2d99-4666-b910-90b3110b7e32-tmp-dir\") pod \"node-resolver-p75ql\" (UID: \"1820ff41-2d99-4666-b910-90b3110b7e32\") " pod="openshift-dns/node-resolver-p75ql"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732518    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-node-log\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732553    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68f721b5-75ba-441c-a277-cb34adc92f53-host-run-netns\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732591    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2948a7e0-5979-4656-9e00-c2cdde279ab2-cnibin\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:34.732691    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:58:34.734418 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:34.732755    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret podName:191c77b5-4225-4b5d-9402-ccdc1765dde8 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:35.232728381 +0000 UTC m=+3.150704830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret") pod "global-pull-secret-syncer-7n4kx" (UID: "191c77b5-4225-4b5d-9402-ccdc1765dde8") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:58:34.735313 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.732903    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2948a7e0-5979-4656-9e00-c2cdde279ab2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.735313 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.734493    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/805523c8-888a-45cb-8291-6ee665e8eeb1-cni-binary-copy\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.736702 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.736677    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/02ee941b-f7e8-49a2-ad16-bf6c1da015ec-konnectivity-ca\") pod \"konnectivity-agent-k5gj5\" (UID: \"02ee941b-f7e8-49a2-ad16-bf6c1da015ec\") " pod="kube-system/konnectivity-agent-k5gj5"
Jul 13 21:58:34.736900 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.736877    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81356c52-66f3-4252-bb9a-d62a47236627-tmp\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.737824 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:34.737804    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:58:34.737924 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:34.737833    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:58:34.737924 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:34.737848    2558 projected.go:196] Error preparing data for projected volume kube-api-access-6ccsd for pod openshift-network-diagnostics/network-check-target-csbjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:58:34.737924 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:34.737918    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd podName:2ad8149f-fe74-4aa2-8f43-bef582499a31 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:35.23790166 +0000 UTC m=+3.155878122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6ccsd" (UniqueName: "kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd") pod "network-check-target-csbjt" (UID: "2ad8149f-fe74-4aa2-8f43-bef582499a31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:58:34.739320 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.739292    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/68f721b5-75ba-441c-a277-cb34adc92f53-ovn-node-metrics-cert\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.739859 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.739820    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/02ee941b-f7e8-49a2-ad16-bf6c1da015ec-agent-certs\") pod \"konnectivity-agent-k5gj5\" (UID: \"02ee941b-f7e8-49a2-ad16-bf6c1da015ec\") " pod="kube-system/konnectivity-agent-k5gj5"
Jul 13 21:58:34.740170 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.740144    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn5zx\" (UniqueName: \"kubernetes.io/projected/2948a7e0-5979-4656-9e00-c2cdde279ab2-kube-api-access-gn5zx\") pod \"multus-additional-cni-plugins-czcbx\" (UID: \"2948a7e0-5979-4656-9e00-c2cdde279ab2\") " pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:34.741978 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.741890    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzvk8\" (UniqueName: \"kubernetes.io/projected/113d2c9e-31c0-44f6-b14f-a1e363928d87-kube-api-access-qzvk8\") pod \"aws-ebs-csi-driver-node-bwb26\" (UID: \"113d2c9e-31c0-44f6-b14f-a1e363928d87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.744428 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.744351    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qzvc\" (UniqueName: \"kubernetes.io/projected/1820ff41-2d99-4666-b910-90b3110b7e32-kube-api-access-6qzvc\") pod \"node-resolver-p75ql\" (UID: \"1820ff41-2d99-4666-b910-90b3110b7e32\") " pod="openshift-dns/node-resolver-p75ql"
Jul 13 21:58:34.745354 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.745330    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/81356c52-66f3-4252-bb9a-d62a47236627-etc-tuned\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.746112 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.745856    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h7vj\" (UniqueName: \"kubernetes.io/projected/805523c8-888a-45cb-8291-6ee665e8eeb1-kube-api-access-7h7vj\") pod \"multus-vht6c\" (UID: \"805523c8-888a-45cb-8291-6ee665e8eeb1\") " pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.746112 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.745898    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drljw\" (UniqueName: \"kubernetes.io/projected/a27ef85e-b4e2-4778-855e-7a05bfeb4a00-kube-api-access-drljw\") pod \"node-ca-w98nt\" (UID: \"a27ef85e-b4e2-4778-855e-7a05bfeb4a00\") " pod="openshift-image-registry/node-ca-w98nt"
Jul 13 21:58:34.746112 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.746016    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4ld9\" (UniqueName: \"kubernetes.io/projected/99237eaa-d7e0-42b7-892c-d8ec6fe1239f-kube-api-access-k4ld9\") pod \"iptables-alerter-q4kp4\" (UID: \"99237eaa-d7e0-42b7-892c-d8ec6fe1239f\") " pod="openshift-network-operator/iptables-alerter-q4kp4"
Jul 13 21:58:34.747359 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.747332    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xknk\" (UniqueName: \"kubernetes.io/projected/68f721b5-75ba-441c-a277-cb34adc92f53-kube-api-access-7xknk\") pod \"ovnkube-node-x6k99\" (UID: \"68f721b5-75ba-441c-a277-cb34adc92f53\") " pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.747656 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.747461    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm58v\" (UniqueName: \"kubernetes.io/projected/81356c52-66f3-4252-bb9a-d62a47236627-kube-api-access-wm58v\") pod \"tuned-zkwrx\" (UID: \"81356c52-66f3-4252-bb9a-d62a47236627\") " pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.748427 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.748403    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dqwx\" (UniqueName: \"kubernetes.io/projected/b485735e-e460-453b-8503-c034e753bf78-kube-api-access-5dqwx\") pod \"network-metrics-daemon-bx7hm\" (UID: \"b485735e-e460-453b-8503-c034e753bf78\") " pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:34.834177 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.834149    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vht6c"
Jul 13 21:58:34.842888 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.842866    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q4kp4"
Jul 13 21:58:34.856336 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.856318    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26"
Jul 13 21:58:34.864186 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.864157    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zkwrx"
Jul 13 21:58:34.871950 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.871931    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p75ql"
Jul 13 21:58:34.879678 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.879662    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:34.888352 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.888332    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k5gj5"
Jul 13 21:58:34.896011 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.895992    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w98nt"
Jul 13 21:58:34.901604 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:34.901582    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-czcbx"
Jul 13 21:58:35.235958 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:35.235872    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:35.236133 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:35.235956    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs\") pod \"network-metrics-daemon-bx7hm\" (UID: \"b485735e-e460-453b-8503-c034e753bf78\") " pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:35.236133 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:35.236055    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:58:35.236133 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:35.236091    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:58:35.236264 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:35.236139    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret podName:191c77b5-4225-4b5d-9402-ccdc1765dde8 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:36.236119074 +0000 UTC m=+4.154095525 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret") pod "global-pull-secret-syncer-7n4kx" (UID: "191c77b5-4225-4b5d-9402-ccdc1765dde8") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:58:35.236264 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:35.236158    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs podName:b485735e-e460-453b-8503-c034e753bf78 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:36.236150405 +0000 UTC m=+4.154126849 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs") pod "network-metrics-daemon-bx7hm" (UID: "b485735e-e460-453b-8503-c034e753bf78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:58:35.336652 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:35.336613    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccsd\" (UniqueName: \"kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd\") pod \"network-check-target-csbjt\" (UID: \"2ad8149f-fe74-4aa2-8f43-bef582499a31\") " pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:35.336819 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:35.336781    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:58:35.336819 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:35.336806    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:58:35.336819 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:35.336817    2558 projected.go:196] Error preparing data for projected volume kube-api-access-6ccsd for pod openshift-network-diagnostics/network-check-target-csbjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:58:35.336955 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:35.336871    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd podName:2ad8149f-fe74-4aa2-8f43-bef582499a31 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:36.336856257 +0000 UTC m=+4.254832702 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6ccsd" (UniqueName: "kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd") pod "network-check-target-csbjt" (UID: "2ad8149f-fe74-4aa2-8f43-bef582499a31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:58:35.674019 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:35.673915    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:53:34 +0000 UTC" deadline="2028-03-27 00:44:16.027897124 +0000 UTC"
Jul 13 21:58:35.674019 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:35.673951    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14930h45m40.353949774s"
Jul 13 21:58:35.832959 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:35.832929    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2948a7e0_5979_4656_9e00_c2cdde279ab2.slice/crio-dc5082e48d77e29279c9848dd41a065a80b4d1b2070598d913980244f34eadd7 WatchSource:0}: Error finding container dc5082e48d77e29279c9848dd41a065a80b4d1b2070598d913980244f34eadd7: Status 404 returned error can't find the container with id dc5082e48d77e29279c9848dd41a065a80b4d1b2070598d913980244f34eadd7
Jul 13 21:58:35.834462 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:35.834439    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81356c52_66f3_4252_bb9a_d62a47236627.slice/crio-b7e123e609a6b97430cebc4b3c7ca1749ecb53db9384e9986721b12f7c571cee WatchSource:0}: Error finding container b7e123e609a6b97430cebc4b3c7ca1749ecb53db9384e9986721b12f7c571cee: Status 404 returned error can't find the container with id b7e123e609a6b97430cebc4b3c7ca1749ecb53db9384e9986721b12f7c571cee
Jul 13 21:58:35.835583 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:35.835544    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1820ff41_2d99_4666_b910_90b3110b7e32.slice/crio-641da94bafdd6461e74748b56573e4f09fdb042fc3f01a28a13ced67c07b72ab WatchSource:0}: Error finding container 641da94bafdd6461e74748b56573e4f09fdb042fc3f01a28a13ced67c07b72ab: Status 404 returned error can't find the container with id 641da94bafdd6461e74748b56573e4f09fdb042fc3f01a28a13ced67c07b72ab
Jul 13 21:58:35.839159 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:35.839128    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f721b5_75ba_441c_a277_cb34adc92f53.slice/crio-eb04ad8e98898357542e92e3f8d171b75a5b2a05a39fa5021b535a0baae1dc47 WatchSource:0}: Error finding container eb04ad8e98898357542e92e3f8d171b75a5b2a05a39fa5021b535a0baae1dc47: Status 404 returned error can't find the container with id eb04ad8e98898357542e92e3f8d171b75a5b2a05a39fa5021b535a0baae1dc47
Jul 13 21:58:35.839939 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:35.839893    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805523c8_888a_45cb_8291_6ee665e8eeb1.slice/crio-a71d6e78b81def11c8412f343e2112f07f76c035ee5901a8e77defb0c327e532 WatchSource:0}: Error finding container a71d6e78b81def11c8412f343e2112f07f76c035ee5901a8e77defb0c327e532: Status 404 returned error can't find the container with id a71d6e78b81def11c8412f343e2112f07f76c035ee5901a8e77defb0c327e532
Jul 13 21:58:35.841500 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:35.841477    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02ee941b_f7e8_49a2_ad16_bf6c1da015ec.slice/crio-69204e0c18d3029bedc5492c6a619d718f9ad785a4e24bf0413f48f54da6679b WatchSource:0}: Error finding container 69204e0c18d3029bedc5492c6a619d718f9ad785a4e24bf0413f48f54da6679b: Status 404 returned error can't find the container with id 69204e0c18d3029bedc5492c6a619d718f9ad785a4e24bf0413f48f54da6679b
Jul 13 21:58:35.842466 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:35.842429    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99237eaa_d7e0_42b7_892c_d8ec6fe1239f.slice/crio-c55de37a8ab1785db643eef1a569ed4edd24acb55e3d129b0530c71a6c6091ef WatchSource:0}: Error finding container c55de37a8ab1785db643eef1a569ed4edd24acb55e3d129b0530c71a6c6091ef: Status 404 returned error can't find the container with id c55de37a8ab1785db643eef1a569ed4edd24acb55e3d129b0530c71a6c6091ef
Jul 13 21:58:35.844148 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:35.844124    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113d2c9e_31c0_44f6_b14f_a1e363928d87.slice/crio-ab38155a374e205b61d0a5596689a0227f5a7e3d04022bfe289d3c280a0dd3a7 WatchSource:0}: Error finding container ab38155a374e205b61d0a5596689a0227f5a7e3d04022bfe289d3c280a0dd3a7: Status 404 returned error can't find the container with id ab38155a374e205b61d0a5596689a0227f5a7e3d04022bfe289d3c280a0dd3a7
Jul 13 21:58:35.846153 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:58:35.846130    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27ef85e_b4e2_4778_855e_7a05bfeb4a00.slice/crio-2c140b159a0466d8779d265cc1664d9800aca704f77aa48b07198ac66a2b8857 WatchSource:0}: Error finding container 2c140b159a0466d8779d265cc1664d9800aca704f77aa48b07198ac66a2b8857: Status 404 returned error can't find the container with id 2c140b159a0466d8779d265cc1664d9800aca704f77aa48b07198ac66a2b8857
Jul 13 21:58:35.972553 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:35.972308    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:58:36.245172 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.244347    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:36.245172 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.244401    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs\") pod \"network-metrics-daemon-bx7hm\" (UID: \"b485735e-e460-453b-8503-c034e753bf78\") " pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:36.245172 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:36.244537    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:58:36.245172 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:36.244597    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs podName:b485735e-e460-453b-8503-c034e753bf78 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:38.244578875 +0000 UTC m=+6.162555325 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs") pod "network-metrics-daemon-bx7hm" (UID: "b485735e-e460-453b-8503-c034e753bf78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:58:36.245172 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:36.245044    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:58:36.245172 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:36.245097    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret podName:191c77b5-4225-4b5d-9402-ccdc1765dde8 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:38.245081319 +0000 UTC m=+6.163057788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret") pod "global-pull-secret-syncer-7n4kx" (UID: "191c77b5-4225-4b5d-9402-ccdc1765dde8") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:58:36.345645 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.345585    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccsd\" (UniqueName: \"kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd\") pod \"network-check-target-csbjt\" (UID: \"2ad8149f-fe74-4aa2-8f43-bef582499a31\") " pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:36.345813 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:36.345783    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:58:36.345894 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:36.345814    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:58:36.345894 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:36.345830    2558 projected.go:196] Error preparing data for projected volume kube-api-access-6ccsd for pod openshift-network-diagnostics/network-check-target-csbjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:58:36.346064 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:36.345898    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd podName:2ad8149f-fe74-4aa2-8f43-bef582499a31 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:38.345877455 +0000 UTC m=+6.263853903 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6ccsd" (UniqueName: "kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd") pod "network-check-target-csbjt" (UID: "2ad8149f-fe74-4aa2-8f43-bef582499a31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:58:36.663643 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.662658    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:36.663643 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:36.662811    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:58:36.663643 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.662913    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:36.663643 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:36.663012    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:58:36.663643 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.663106    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:36.663643 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:36.663195    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:58:36.674744 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.674705    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:53:34 +0000 UTC" deadline="2028-03-28 19:27:25.787845232 +0000 UTC"
Jul 13 21:58:36.674744 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.674744    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14973h28m49.113106374s"
Jul 13 21:58:36.679827 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.679796    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w98nt" event={"ID":"a27ef85e-b4e2-4778-855e-7a05bfeb4a00","Type":"ContainerStarted","Data":"2c140b159a0466d8779d265cc1664d9800aca704f77aa48b07198ac66a2b8857"}
Jul 13 21:58:36.691508 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.691149    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k5gj5" event={"ID":"02ee941b-f7e8-49a2-ad16-bf6c1da015ec","Type":"ContainerStarted","Data":"69204e0c18d3029bedc5492c6a619d718f9ad785a4e24bf0413f48f54da6679b"}
Jul 13 21:58:36.699589 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.699557    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q4kp4" event={"ID":"99237eaa-d7e0-42b7-892c-d8ec6fe1239f","Type":"ContainerStarted","Data":"c55de37a8ab1785db643eef1a569ed4edd24acb55e3d129b0530c71a6c6091ef"}
Jul 13 21:58:36.706524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.706493    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vht6c" event={"ID":"805523c8-888a-45cb-8291-6ee665e8eeb1","Type":"ContainerStarted","Data":"a71d6e78b81def11c8412f343e2112f07f76c035ee5901a8e77defb0c327e532"}
Jul 13 21:58:36.714707 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.713869    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p75ql" event={"ID":"1820ff41-2d99-4666-b910-90b3110b7e32","Type":"ContainerStarted","Data":"641da94bafdd6461e74748b56573e4f09fdb042fc3f01a28a13ced67c07b72ab"}
Jul 13 21:58:36.716658 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.716628    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zkwrx" event={"ID":"81356c52-66f3-4252-bb9a-d62a47236627","Type":"ContainerStarted","Data":"b7e123e609a6b97430cebc4b3c7ca1749ecb53db9384e9986721b12f7c571cee"}
Jul 13 21:58:36.719233 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.719183    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czcbx" event={"ID":"2948a7e0-5979-4656-9e00-c2cdde279ab2","Type":"ContainerStarted","Data":"dc5082e48d77e29279c9848dd41a065a80b4d1b2070598d913980244f34eadd7"}
Jul 13 21:58:36.726119 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.726094    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-101.ec2.internal" event={"ID":"3b7f01a0953299e98dac4176c400f9a5","Type":"ContainerStarted","Data":"8514bcce4dda5792698bc479007a220aa1a5ea85b95c35cdbb05be1007649766"}
Jul 13 21:58:36.741295 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.741235    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-101.ec2.internal" podStartSLOduration=2.741215571 podStartE2EDuration="2.741215571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:58:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:58:36.740564921 +0000 UTC m=+4.658541394" watchObservedRunningTime="2026-07-13 21:58:36.741215571 +0000 UTC m=+4.659192039"
Jul 13 21:58:36.747160 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.747034    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26" event={"ID":"113d2c9e-31c0-44f6-b14f-a1e363928d87","Type":"ContainerStarted","Data":"ab38155a374e205b61d0a5596689a0227f5a7e3d04022bfe289d3c280a0dd3a7"}
Jul 13 21:58:36.754435 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.754361    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99" event={"ID":"68f721b5-75ba-441c-a277-cb34adc92f53","Type":"ContainerStarted","Data":"eb04ad8e98898357542e92e3f8d171b75a5b2a05a39fa5021b535a0baae1dc47"}
Jul 13 21:58:36.765638 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.765556    2558 generic.go:356] "Generic (PLEG): container finished" podID="40671ae6695f47aabb33292598c63a42" containerID="4c4399e024de0a0b409d74d30f2ffb0a61413976f6396525e606589b0451658a" exitCode=0
Jul 13 21:58:36.765638 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:36.765594    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal" event={"ID":"40671ae6695f47aabb33292598c63a42","Type":"ContainerDied","Data":"4c4399e024de0a0b409d74d30f2ffb0a61413976f6396525e606589b0451658a"}
Jul 13 21:58:37.788315 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:37.788278    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal" event={"ID":"40671ae6695f47aabb33292598c63a42","Type":"ContainerStarted","Data":"ef3a067f3265b4f5705aee6be6ec7c406a3b3af94e904d168eabfa73ced9ff7d"}
Jul 13 21:58:38.267663 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:38.267572    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:38.267663 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:38.267631    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs\") pod \"network-metrics-daemon-bx7hm\" (UID: \"b485735e-e460-453b-8503-c034e753bf78\") " pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:38.267870 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:38.267784    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:58:38.267870 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:38.267843    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs podName:b485735e-e460-453b-8503-c034e753bf78 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:42.26782628 +0000 UTC m=+10.185802729 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs") pod "network-metrics-daemon-bx7hm" (UID: "b485735e-e460-453b-8503-c034e753bf78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:58:38.268249 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:38.268232    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:58:38.268344 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:38.268286    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret podName:191c77b5-4225-4b5d-9402-ccdc1765dde8 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:42.268269773 +0000 UTC m=+10.186246221 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret") pod "global-pull-secret-syncer-7n4kx" (UID: "191c77b5-4225-4b5d-9402-ccdc1765dde8") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:58:38.368379 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:38.368344    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccsd\" (UniqueName: \"kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd\") pod \"network-check-target-csbjt\" (UID: \"2ad8149f-fe74-4aa2-8f43-bef582499a31\") " pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:38.368561 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:38.368511    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:58:38.368561 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:38.368533    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:58:38.368561 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:38.368558    2558 projected.go:196] Error preparing data for projected volume kube-api-access-6ccsd for pod openshift-network-diagnostics/network-check-target-csbjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:58:38.368723 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:38.368612    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd podName:2ad8149f-fe74-4aa2-8f43-bef582499a31 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:42.368595885 +0000 UTC m=+10.286572341 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6ccsd" (UniqueName: "kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd") pod "network-check-target-csbjt" (UID: "2ad8149f-fe74-4aa2-8f43-bef582499a31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:58:38.663204 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:38.662254    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:38.663204 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:38.662374    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:58:38.663204 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:38.662775    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:38.663204 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:38.662868    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:58:38.663204 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:38.662931    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:38.663204 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:38.663009    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:58:40.662402 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:40.661530    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:40.662402 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:40.661672    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:58:40.662402 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:40.662063    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:40.662402 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:40.662168    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:58:40.662402 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:40.662238    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:40.662402 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:40.662321    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:58:42.302205 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:42.302162    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:42.302671 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:42.302224    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs\") pod \"network-metrics-daemon-bx7hm\" (UID: \"b485735e-e460-453b-8503-c034e753bf78\") " pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:42.302671 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:42.302328    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:58:42.302671 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:42.302377    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:58:42.302671 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:42.302399    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret podName:191c77b5-4225-4b5d-9402-ccdc1765dde8 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:50.302378483 +0000 UTC m=+18.220354935 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret") pod "global-pull-secret-syncer-7n4kx" (UID: "191c77b5-4225-4b5d-9402-ccdc1765dde8") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:58:42.302671 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:42.302446    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs podName:b485735e-e460-453b-8503-c034e753bf78 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:50.302424573 +0000 UTC m=+18.220401030 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs") pod "network-metrics-daemon-bx7hm" (UID: "b485735e-e460-453b-8503-c034e753bf78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:58:42.403822 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:42.403193    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccsd\" (UniqueName: \"kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd\") pod \"network-check-target-csbjt\" (UID: \"2ad8149f-fe74-4aa2-8f43-bef582499a31\") " pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:42.403822 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:42.403389    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:58:42.403822 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:42.403407    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:58:42.403822 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:42.403419    2558 projected.go:196] Error preparing data for projected volume kube-api-access-6ccsd for pod openshift-network-diagnostics/network-check-target-csbjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:58:42.403822 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:42.403476    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd podName:2ad8149f-fe74-4aa2-8f43-bef582499a31 nodeName:}" failed. No retries permitted until 2026-07-13 21:58:50.403457925 +0000 UTC m=+18.321434372 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6ccsd" (UniqueName: "kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd") pod "network-check-target-csbjt" (UID: "2ad8149f-fe74-4aa2-8f43-bef582499a31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:58:42.661167 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:42.661015    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:42.661167 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:42.661122    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:58:42.661552 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:42.661472    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:42.661660 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:42.661599    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:58:42.661660 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:42.661633    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:42.661792 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:42.661710    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:58:44.660584 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:44.660081    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:44.660584 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:44.660212    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:44.660584 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:44.660224    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:58:44.660584 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:44.660332    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:58:44.660584 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:44.660382    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:44.660584 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:44.660455    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:58:46.660908 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:46.660875    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:46.661337 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:46.660876    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:46.661337 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:46.660984    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:58:46.661337 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:46.660999    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:46.661337 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:46.661116    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:58:46.661337 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:46.661230    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:58:48.660293 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:48.660251    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:48.660773 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:48.660264    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:48.660773 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:48.660376    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:58:48.660773 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:48.660267    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:48.660773 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:48.660455    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:58:48.660773 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:48.660589    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:58:50.365239 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:50.365211    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:50.365239 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:50.365244    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs\") pod \"network-metrics-daemon-bx7hm\" (UID: \"b485735e-e460-453b-8503-c034e753bf78\") " pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:50.365795 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:50.365361    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:58:50.365795 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:50.365399    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:58:50.365795 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:50.365430    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs podName:b485735e-e460-453b-8503-c034e753bf78 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:06.365400756 +0000 UTC m=+34.283377199 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs") pod "network-metrics-daemon-bx7hm" (UID: "b485735e-e460-453b-8503-c034e753bf78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:58:50.365795 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:50.365455    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret podName:191c77b5-4225-4b5d-9402-ccdc1765dde8 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:06.365439422 +0000 UTC m=+34.283415864 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret") pod "global-pull-secret-syncer-7n4kx" (UID: "191c77b5-4225-4b5d-9402-ccdc1765dde8") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:58:50.465555 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:50.465505    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccsd\" (UniqueName: \"kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd\") pod \"network-check-target-csbjt\" (UID: \"2ad8149f-fe74-4aa2-8f43-bef582499a31\") " pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:50.465742 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:50.465678    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:58:50.465742 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:50.465707    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:58:50.465742 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:50.465736    2558 projected.go:196] Error preparing data for projected volume kube-api-access-6ccsd for pod openshift-network-diagnostics/network-check-target-csbjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:58:50.465949 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:50.465790    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd podName:2ad8149f-fe74-4aa2-8f43-bef582499a31 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:06.465771337 +0000 UTC m=+34.383747782 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6ccsd" (UniqueName: "kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd") pod "network-check-target-csbjt" (UID: "2ad8149f-fe74-4aa2-8f43-bef582499a31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:58:50.660535 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:50.660446    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:50.660535 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:50.660459    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:50.660752 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:50.660577    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:58:50.660752 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:50.660651    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:58:50.660752 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:50.660716    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:50.660852 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:50.660828    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:58:52.660618 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:52.660587    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:52.661028 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:52.660689    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:58:52.661028 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:52.660772    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:52.661028 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:52.660880    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:58:52.661028 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:52.660927    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:52.661028 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:52.660991    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:58:54.660052 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.659890    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:54.660517 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.659892    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:54.660517 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.659932    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:54.660517 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:54.660131    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:58:54.660517 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:54.660256    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:58:54.660517 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:54.660378    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:58:54.824980 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.824958    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 21:58:54.825316 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.825291    2558 generic.go:356] "Generic (PLEG): container finished" podID="68f721b5-75ba-441c-a277-cb34adc92f53" containerID="e5f6c2f33f6aba307643fe7d597726daa31a4e760a56e5c9efe8a7e74e987389" exitCode=1
Jul 13 21:58:54.825393 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.825364    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99" event={"ID":"68f721b5-75ba-441c-a277-cb34adc92f53","Type":"ContainerStarted","Data":"e1271f8bc3422aaca14db7611b7e444a0296be4b390e229f91748c6fd05968ad"}
Jul 13 21:58:54.825472 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.825458    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99" event={"ID":"68f721b5-75ba-441c-a277-cb34adc92f53","Type":"ContainerDied","Data":"e5f6c2f33f6aba307643fe7d597726daa31a4e760a56e5c9efe8a7e74e987389"}
Jul 13 21:58:54.825525 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.825513    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99" event={"ID":"68f721b5-75ba-441c-a277-cb34adc92f53","Type":"ContainerStarted","Data":"0f0e944bc44ba4c6848eebcdc56b5d7df03f69b5341e10960a8f4673939667a9"}
Jul 13 21:58:54.827057 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.827011    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w98nt" event={"ID":"a27ef85e-b4e2-4778-855e-7a05bfeb4a00","Type":"ContainerStarted","Data":"ed64cf3bb4ebb083efe5b706bd4b449854d9feed1e616483b88febc34b32dbb6"}
Jul 13 21:58:54.828310 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.828289    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k5gj5" event={"ID":"02ee941b-f7e8-49a2-ad16-bf6c1da015ec","Type":"ContainerStarted","Data":"488b4ce946815a57e9575e78c2e324ac68a342af145c77f3e51df7a74b19db7e"}
Jul 13 21:58:54.831935 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.831914    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vht6c" event={"ID":"805523c8-888a-45cb-8291-6ee665e8eeb1","Type":"ContainerStarted","Data":"8ea8a18dc14705cda02af5dad5a18b6666212d0620a0e3e38eb4602907dc6a2a"}
Jul 13 21:58:54.833317 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.833300    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p75ql" event={"ID":"1820ff41-2d99-4666-b910-90b3110b7e32","Type":"ContainerStarted","Data":"91a4f8da57d3545939e610a0409e35e30d6d30aa4ca86de32825aa76b188dfda"}
Jul 13 21:58:54.834761 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.834738    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zkwrx" event={"ID":"81356c52-66f3-4252-bb9a-d62a47236627","Type":"ContainerStarted","Data":"7fdaba9a87f12173c3414dcf75e553f4ac6fcc2b99be5f72ff036731115e95e5"}
Jul 13 21:58:54.836431 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.836379    2558 generic.go:356] "Generic (PLEG): container finished" podID="2948a7e0-5979-4656-9e00-c2cdde279ab2" containerID="aaab621453fd3b71686de330d8b60dcac717574f3de043cf34fee751d196be07" exitCode=0
Jul 13 21:58:54.836496 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.836458    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czcbx" event={"ID":"2948a7e0-5979-4656-9e00-c2cdde279ab2","Type":"ContainerDied","Data":"aaab621453fd3b71686de330d8b60dcac717574f3de043cf34fee751d196be07"}
Jul 13 21:58:54.838500 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.838483    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26" event={"ID":"113d2c9e-31c0-44f6-b14f-a1e363928d87","Type":"ContainerStarted","Data":"462e394f58de08ed0e0f06360a938e509d5c0b90da5d2082e21e68a6fbdd23cc"}
Jul 13 21:58:54.843439 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.843404    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-101.ec2.internal" podStartSLOduration=20.84339247 podStartE2EDuration="20.84339247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:58:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:58:37.805598173 +0000 UTC m=+5.723574643" watchObservedRunningTime="2026-07-13 21:58:54.84339247 +0000 UTC m=+22.761368932"
Jul 13 21:58:54.843618 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.843598    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w98nt" podStartSLOduration=3.386329545 podStartE2EDuration="21.843592764s" totalImagesPullingTime="18.457263219s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:58:54.843098757 +0000 UTC m=+22.761075221" watchObservedRunningTime="2026-07-13 21:58:54.843592764 +0000 UTC m=+22.761569228"
Jul 13 21:58:54.860301 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.860185    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vht6c" podStartSLOduration=3.33379172 podStartE2EDuration="21.860171798s" totalImagesPullingTime="18.526380078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:58:54.859959667 +0000 UTC m=+22.777936134" watchObservedRunningTime="2026-07-13 21:58:54.860171798 +0000 UTC m=+22.778148263"
Jul 13 21:58:54.875345 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.875313    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zkwrx" podStartSLOduration=3.387404585 podStartE2EDuration="21.875301592s" totalImagesPullingTime="18.487897007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:58:54.875094847 +0000 UTC m=+22.793071311" watchObservedRunningTime="2026-07-13 21:58:54.875301592 +0000 UTC m=+22.793278056"
Jul 13 21:58:54.888880 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.888840    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-p75ql" podStartSLOduration=3.4202246130000002 podStartE2EDuration="21.888829296s" totalImagesPullingTime="18.468604683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:58:54.888552677 +0000 UTC m=+22.806529135" watchObservedRunningTime="2026-07-13 21:58:54.888829296 +0000 UTC m=+22.806805758"
Jul 13 21:58:54.902598 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:54.902568    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-k5gj5" podStartSLOduration=11.215311721 podStartE2EDuration="21.902528655s" totalImagesPullingTime="10.687216934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:58:54.902313696 +0000 UTC m=+22.820290160" watchObservedRunningTime="2026-07-13 21:58:54.902528655 +0000 UTC m=+22.820505176"
Jul 13 21:58:55.089283 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:55.089257    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:55.145705 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:55.145681    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:55.524972 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:55.524936    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:55.622363 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:55.622281    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:55.752585 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:55.752555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:55.842477 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:55.842445    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q4kp4" event={"ID":"99237eaa-d7e0-42b7-892c-d8ec6fe1239f","Type":"ContainerStarted","Data":"226f5f6731859754fdff3452146b420341046e9d4142c22e2a3443fd9b3e9da2"}
Jul 13 21:58:55.845571 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:55.845535    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 21:58:55.846112 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:55.846082    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99" event={"ID":"68f721b5-75ba-441c-a277-cb34adc92f53","Type":"ContainerStarted","Data":"6b5bde8ccef687add1054a34453838a245708dddef168976da4e4e4c42d1e105"}
Jul 13 21:58:55.846195 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:55.846162    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99" event={"ID":"68f721b5-75ba-441c-a277-cb34adc92f53","Type":"ContainerStarted","Data":"594c3142d6e5fda761e600d711f8ec229fd8fdf2d7e358ff2fe739ff316b8aa9"}
Jul 13 21:58:55.846264 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:55.846210    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99" event={"ID":"68f721b5-75ba-441c-a277-cb34adc92f53","Type":"ContainerStarted","Data":"fbe428f92045e074d302a5e67414c48163cf4bb43ca72eac66d582e3ae169944"}
Jul 13 21:58:55.969060 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:55.969034    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 13 21:58:56.487751 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:56.487718    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:56.592605 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:56.592469    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T21:58:55.969053536Z","UUID":"f6b8c265-d2d8-4f73-bf24-15f6bd031e8d","Handler":null,"Name":"","Endpoint":""}
Jul 13 21:58:56.594472 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:56.594451    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 13 21:58:56.594675 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:56.594501    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 13 21:58:56.600156 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:56.600128    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:56.660222 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:56.660192    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:56.660375 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:56.660211    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:56.660375 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:56.660222    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:56.660375 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:56.660319    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:58:56.660514 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:56.660402    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:58:56.660514 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:56.660494    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:58:56.689689 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:56.689651    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:56.850644 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:56.850575    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26" event={"ID":"113d2c9e-31c0-44f6-b14f-a1e363928d87","Type":"ContainerStarted","Data":"0ae399d730d868e359088bcee06b2a28e21caeeb284c305b21e1c6eb0ab0a3da"}
Jul 13 21:58:56.875127 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:56.875102    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:56.975401 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:56.975370    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:57.081850 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:57.081805    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:57.224667 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:57.224597    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:57.855212 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:57.855032    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26" event={"ID":"113d2c9e-31c0-44f6-b14f-a1e363928d87","Type":"ContainerStarted","Data":"50337014d378eca637fab877f46c07c6ee52d1996c8d7919fc2ab0619861aa7a"}
Jul 13 21:58:57.858511 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:57.858491    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 21:58:57.858909 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:57.858889    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99" event={"ID":"68f721b5-75ba-441c-a277-cb34adc92f53","Type":"ContainerStarted","Data":"ee80000788a11f13284107e6725dca3d6b12b277b4eb074d5d40e928bf91c1e5"}
Jul 13 21:58:57.884458 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:57.884411    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-q4kp4" podStartSLOduration=6.423309099 podStartE2EDuration="24.884393673s" totalImagesPullingTime="18.461084574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:58:55.860639215 +0000 UTC m=+23.778615688" watchObservedRunningTime="2026-07-13 21:58:57.884393673 +0000 UTC m=+25.802370137"
Jul 13 21:58:57.884758 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:57.884731    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwb26" podStartSLOduration=4.195636663 podStartE2EDuration="24.88471707s" totalImagesPullingTime="20.689080407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:58:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:58:57.88457441 +0000 UTC m=+25.802550874" watchObservedRunningTime="2026-07-13 21:58:57.88471707 +0000 UTC m=+25.802693536"
Jul 13 21:58:58.003450 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:58.003417    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-k5gj5"
Jul 13 21:58:58.004522 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:58.004503    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-k5gj5"
Jul 13 21:58:58.048342 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:58.048303    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:58.423076 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:58.423046    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:58.580083 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:58.580049    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:58.660586 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:58.660562    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:58:58.660743 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:58.660561    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:58:58.660743 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:58.660673    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:58:58.660743 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:58.660562    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:58:58.660900 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:58.660746    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:58:58.660900 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:58:58.660840    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:58:58.765458 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:58.765401    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:58.861710 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:58.861658    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-k5gj5"
Jul 13 21:58:58.862203 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:58.862022    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-k5gj5"
Jul 13 21:58:58.972028 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:58.971999    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:59.054904 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:59.054833    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:59.586941 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:59.586919    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:58:59.866828 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:59.866614    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 21:58:59.867454 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:59.867103    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99" event={"ID":"68f721b5-75ba-441c-a277-cb34adc92f53","Type":"ContainerStarted","Data":"d17d3192f27015e666a320151549c01427c3d5f05e40253b94bb6e5f740d82d2"}
Jul 13 21:58:59.867454 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:59.867326    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:59.867454 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:59.867400    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:59.867627 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:59.867523    2558 scope.go:117] "RemoveContainer" containerID="e5f6c2f33f6aba307643fe7d597726daa31a4e760a56e5c9efe8a7e74e987389"
Jul 13 21:58:59.869062 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:59.869030    2558 generic.go:356] "Generic (PLEG): container finished" podID="2948a7e0-5979-4656-9e00-c2cdde279ab2" containerID="813af10c517c8acf720f49346bf176d11e9bc87d45420852b556739f4e0b4615" exitCode=0
Jul 13 21:58:59.869189 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:59.869083    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czcbx" event={"ID":"2948a7e0-5979-4656-9e00-c2cdde279ab2","Type":"ContainerDied","Data":"813af10c517c8acf720f49346bf176d11e9bc87d45420852b556739f4e0b4615"}
Jul 13 21:58:59.884605 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:59.884587    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:58:59.965180 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:58:59.965157    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:00.031110 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.031085    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:00.178314 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.178229    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:00.281016 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.280988    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:00.380931 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.380898    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:00.481300 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.481273    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:00.582785 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.582754    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:00.636582 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.636554    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:00.660705 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.660680    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:00.660821 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.660680    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:00.660821 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:00.660776    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:00.660940 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.660680    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:00.660940 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:00.660856    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:00.661031 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:00.660968    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:00.784075 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.784020    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:00.874501 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.874481    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 21:59:00.874890 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.874827    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99" event={"ID":"68f721b5-75ba-441c-a277-cb34adc92f53","Type":"ContainerStarted","Data":"0b79fc27e5aec6ab40f21e59452107bf40360f6c6096e970722cbfd0bc4bf78f"}
Jul 13 21:59:00.875189 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.875171    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:59:00.886919 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.886898    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:00.890066 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.890052    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:59:00.903882 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:00.903843    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99" podStartSLOduration=9.356843448 podStartE2EDuration="27.903827552s" totalImagesPullingTime="18.546984104s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:00.903679361 +0000 UTC m=+28.821655828" watchObservedRunningTime="2026-07-13 21:59:00.903827552 +0000 UTC m=+28.821804013"
Jul 13 21:59:01.878955 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:01.878926    2558 generic.go:356] "Generic (PLEG): container finished" podID="2948a7e0-5979-4656-9e00-c2cdde279ab2" containerID="11453bdc09f896715f2b902d4a2234d8bdd2e9eb552b12c867301bce33750459" exitCode=0
Jul 13 21:59:01.879461 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:01.879004    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czcbx" event={"ID":"2948a7e0-5979-4656-9e00-c2cdde279ab2","Type":"ContainerDied","Data":"11453bdc09f896715f2b902d4a2234d8bdd2e9eb552b12c867301bce33750459"}
Jul 13 21:59:01.974685 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:01.974654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:02.187928 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:02.187862    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:02.373938 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:02.373907    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:02.581216 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:02.581190    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:02.661436 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:02.661401    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:02.661609 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:02.661500    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:02.661609 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:02.661581    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:02.661750 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:02.661669    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:02.661750 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:02.661734    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:02.661847 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:02.661788    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:02.686739 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:02.686709    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:02.786447 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:02.786423    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log" limit=1024
Jul 13 21:59:03.884424 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:03.884392    2558 generic.go:356] "Generic (PLEG): container finished" podID="2948a7e0-5979-4656-9e00-c2cdde279ab2" containerID="76ab9953cd9db786b228bb67ada47f9e5eb9e9a626485701d4775e0179911405" exitCode=0
Jul 13 21:59:03.884769 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:03.884436    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czcbx" event={"ID":"2948a7e0-5979-4656-9e00-c2cdde279ab2","Type":"ContainerDied","Data":"76ab9953cd9db786b228bb67ada47f9e5eb9e9a626485701d4775e0179911405"}
Jul 13 21:59:04.659961 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:04.659934    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:04.659961 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:04.659943    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:04.660184 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:04.660022    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:04.660184 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:04.660110    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:04.660265 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:04.660101    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:04.660265 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:04.660190    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:06.383384 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:06.383347    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:06.383384 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:06.383389    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs\") pod \"network-metrics-daemon-bx7hm\" (UID: \"b485735e-e460-453b-8503-c034e753bf78\") " pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:06.383855 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:06.383492    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:59:06.383855 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:06.383494    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:59:06.383855 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:06.383556    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs podName:b485735e-e460-453b-8503-c034e753bf78 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:38.383525649 +0000 UTC m=+66.301502093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs") pod "network-metrics-daemon-bx7hm" (UID: "b485735e-e460-453b-8503-c034e753bf78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:59:06.383855 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:06.383571    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret podName:191c77b5-4225-4b5d-9402-ccdc1765dde8 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:38.383564649 +0000 UTC m=+66.301541091 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret") pod "global-pull-secret-syncer-7n4kx" (UID: "191c77b5-4225-4b5d-9402-ccdc1765dde8") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:59:06.484264 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:06.484229    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccsd\" (UniqueName: \"kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd\") pod \"network-check-target-csbjt\" (UID: \"2ad8149f-fe74-4aa2-8f43-bef582499a31\") " pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:06.484412 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:06.484356    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:59:06.484412 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:06.484371    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:59:06.484412 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:06.484383    2558 projected.go:196] Error preparing data for projected volume kube-api-access-6ccsd for pod openshift-network-diagnostics/network-check-target-csbjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:59:06.484535 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:06.484428    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd podName:2ad8149f-fe74-4aa2-8f43-bef582499a31 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:38.484416266 +0000 UTC m=+66.402392708 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6ccsd" (UniqueName: "kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd") pod "network-check-target-csbjt" (UID: "2ad8149f-fe74-4aa2-8f43-bef582499a31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:59:06.660880 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:06.660751    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:06.660880 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:06.660760    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:06.661082 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:06.660881    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:06.661082 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:06.660894    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:06.661082 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:06.661008    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:06.661236 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:06.661104    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:08.660721 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:08.660697    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:08.660721 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:08.660713    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:08.661131 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:08.660835    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:08.661131 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:08.660884    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:08.661131 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:08.660931    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:08.661131 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:08.660988    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:08.897523 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:08.897499    2558 generic.go:356] "Generic (PLEG): container finished" podID="2948a7e0-5979-4656-9e00-c2cdde279ab2" containerID="f7d97b15b56a761729baed4f6389fec16053b5e005728fda4dd7eb25504a7e59" exitCode=0
Jul 13 21:59:08.897672 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:08.897572    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czcbx" event={"ID":"2948a7e0-5979-4656-9e00-c2cdde279ab2","Type":"ContainerDied","Data":"f7d97b15b56a761729baed4f6389fec16053b5e005728fda4dd7eb25504a7e59"}
Jul 13 21:59:09.902470 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:09.902233    2558 generic.go:356] "Generic (PLEG): container finished" podID="2948a7e0-5979-4656-9e00-c2cdde279ab2" containerID="cc63b4529f4b7e806914206ae185390afbe1e5b353d45c41390eae94b11e5915" exitCode=0
Jul 13 21:59:09.902853 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:09.902307    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czcbx" event={"ID":"2948a7e0-5979-4656-9e00-c2cdde279ab2","Type":"ContainerDied","Data":"cc63b4529f4b7e806914206ae185390afbe1e5b353d45c41390eae94b11e5915"}
Jul 13 21:59:10.660450 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:10.660417    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:10.660645 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:10.660421    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:10.660645 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:10.660533    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:10.660645 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:10.660525    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:10.660797 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:10.660644    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:10.660797 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:10.660739    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:10.907171 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:10.907131    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czcbx" event={"ID":"2948a7e0-5979-4656-9e00-c2cdde279ab2","Type":"ContainerStarted","Data":"4a59f02ca7c483ed2e13d42cabca2ffc8902e019999c33ef7898c8198edf8d37"}
Jul 13 21:59:10.945431 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:10.945358    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-czcbx" podStartSLOduration=8.417575225 podStartE2EDuration="37.945343807s" totalImagesPullingTime="29.527768582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:58:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:10.944994777 +0000 UTC m=+38.862971247" watchObservedRunningTime="2026-07-13 21:59:10.945343807 +0000 UTC m=+38.863320272"
Jul 13 21:59:12.661231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:12.661206    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:12.661575 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:12.661276    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:12.661575 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:12.661371    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:12.661575 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:12.661451    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:12.661575 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:12.661458    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:12.661575 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:12.661506    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:14.660713 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:14.660685    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:14.661068 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:14.660689    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:14.661068 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:14.660777    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:14.661068 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:14.660696    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:14.661068 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:14.660872    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:14.661068 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:14.660956    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:16.660081 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:16.660046    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:16.660081 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:16.660061    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:16.660573 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:16.660046    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:16.660573 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:16.660149    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:16.660573 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:16.660255    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:16.660573 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:16.660342    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:18.663466 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:18.663438    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:18.664054 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:18.663535    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:18.664054 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:18.663563    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:18.664054 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:18.663668    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:18.664054 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:18.663689    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:18.664054 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:18.663753    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:20.663486 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:20.663462    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:20.663486 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:20.663475    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:20.664119 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:20.663536    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:20.664119 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:20.663559    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:20.664119 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:20.663632    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:20.664119 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:20.663707    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:20.680065 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:20.680042    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7n4kx"]
Jul 13 21:59:20.683740 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:20.683720    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bx7hm"]
Jul 13 21:59:20.684355 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:20.684336    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-csbjt"]
Jul 13 21:59:20.930619 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:20.930588    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:20.930619 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:20.930598    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:20.930619 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:20.930597    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:20.930823 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:20.930680    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:20.930823 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:20.930743    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:20.930890 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:20.930818    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:22.660987 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:22.660956    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:22.661439 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:22.661050    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:22.661439 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:22.661122    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:22.661439 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:22.661207    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:22.661439 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:22.661216    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:22.661439 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:22.661283    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:24.660117 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:24.659933    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:24.660528 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:24.659936    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:24.660528 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:24.660179    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-csbjt" podUID="2ad8149f-fe74-4aa2-8f43-bef582499a31"
Jul 13 21:59:24.660528 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:24.659953    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:24.660528 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:24.660252    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7n4kx" podUID="191c77b5-4225-4b5d-9402-ccdc1765dde8"
Jul 13 21:59:24.660528 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:24.660372    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx7hm" podUID="b485735e-e460-453b-8503-c034e753bf78"
Jul 13 21:59:25.413416 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.413393    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-101.ec2.internal" event="NodeReady"
Jul 13 21:59:25.413661 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.413494    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 13 21:59:25.456870 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.456842    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b57f6"]
Jul 13 21:59:25.473024 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.473004    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wjjwx"]
Jul 13 21:59:25.473147 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.473131    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.476062 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.476043    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jqfwq\""
Jul 13 21:59:25.476198 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.476059    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 13 21:59:25.476198 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.476043    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 13 21:59:25.493480 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.493235    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b57f6"]
Jul 13 21:59:25.493480 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.493279    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wjjwx"]
Jul 13 21:59:25.493480 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.493301    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jsl8d"]
Jul 13 21:59:25.493480 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.493400    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wjjwx"
Jul 13 21:59:25.496002 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.495980    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 13 21:59:25.496104 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.496051    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7pv2b\""
Jul 13 21:59:25.496104 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.496055    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 13 21:59:25.496214 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.496113    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 13 21:59:25.507364 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.507341    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jsl8d"]
Jul 13 21:59:25.507552 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.507455    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.509936 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.509815    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 13 21:59:25.509936 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.509843    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tlwrc\""
Jul 13 21:59:25.509936 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.509858    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 13 21:59:25.509936 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.509887    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 13 21:59:25.510187 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.510102    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 13 21:59:25.522725 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.522709    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-data-volume\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.522811 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.522733    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxmmj\" (UniqueName: \"kubernetes.io/projected/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-kube-api-access-lxmmj\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.522811 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.522749    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e82d0f20-1cb3-4f5f-a04e-3e10b5f76593-metrics-tls\") pod \"dns-default-b57f6\" (UID: \"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593\") " pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.522811 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.522773    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e82d0f20-1cb3-4f5f-a04e-3e10b5f76593-tmp-dir\") pod \"dns-default-b57f6\" (UID: \"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593\") " pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.522955 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.522851    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-crio-socket\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.522955 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.522917    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e82d0f20-1cb3-4f5f-a04e-3e10b5f76593-config-volume\") pod \"dns-default-b57f6\" (UID: \"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593\") " pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.523025 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.522956    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7d9e375-8217-427d-81d9-ae0aaa546025-cert\") pod \"ingress-canary-wjjwx\" (UID: \"a7d9e375-8217-427d-81d9-ae0aaa546025\") " pod="openshift-ingress-canary/ingress-canary-wjjwx"
Jul 13 21:59:25.523025 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.522995    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.523025 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.523021    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p46hn\" (UniqueName: \"kubernetes.io/projected/a7d9e375-8217-427d-81d9-ae0aaa546025-kube-api-access-p46hn\") pod \"ingress-canary-wjjwx\" (UID: \"a7d9e375-8217-427d-81d9-ae0aaa546025\") " pod="openshift-ingress-canary/ingress-canary-wjjwx"
Jul 13 21:59:25.523177 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.523042    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkxrt\" (UniqueName: \"kubernetes.io/projected/e82d0f20-1cb3-4f5f-a04e-3e10b5f76593-kube-api-access-mkxrt\") pod \"dns-default-b57f6\" (UID: \"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593\") " pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.523177 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.523070    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.624133 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624112    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e82d0f20-1cb3-4f5f-a04e-3e10b5f76593-config-volume\") pod \"dns-default-b57f6\" (UID: \"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593\") " pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.624212 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624135    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7d9e375-8217-427d-81d9-ae0aaa546025-cert\") pod \"ingress-canary-wjjwx\" (UID: \"a7d9e375-8217-427d-81d9-ae0aaa546025\") " pod="openshift-ingress-canary/ingress-canary-wjjwx"
Jul 13 21:59:25.624212 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624151    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.624212 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624167    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p46hn\" (UniqueName: \"kubernetes.io/projected/a7d9e375-8217-427d-81d9-ae0aaa546025-kube-api-access-p46hn\") pod \"ingress-canary-wjjwx\" (UID: \"a7d9e375-8217-427d-81d9-ae0aaa546025\") " pod="openshift-ingress-canary/ingress-canary-wjjwx"
Jul 13 21:59:25.624212 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624192    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkxrt\" (UniqueName: \"kubernetes.io/projected/e82d0f20-1cb3-4f5f-a04e-3e10b5f76593-kube-api-access-mkxrt\") pod \"dns-default-b57f6\" (UID: \"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593\") " pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.624400 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624367    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.624451 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624437    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-data-volume\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.624502 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624462    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxmmj\" (UniqueName: \"kubernetes.io/projected/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-kube-api-access-lxmmj\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.624502 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624489    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e82d0f20-1cb3-4f5f-a04e-3e10b5f76593-metrics-tls\") pod \"dns-default-b57f6\" (UID: \"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593\") " pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.624624 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624514    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e82d0f20-1cb3-4f5f-a04e-3e10b5f76593-tmp-dir\") pod \"dns-default-b57f6\" (UID: \"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593\") " pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.624925 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624745    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-crio-socket\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.624925 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624804    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e82d0f20-1cb3-4f5f-a04e-3e10b5f76593-tmp-dir\") pod \"dns-default-b57f6\" (UID: \"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593\") " pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.624925 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624837    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e82d0f20-1cb3-4f5f-a04e-3e10b5f76593-config-volume\") pod \"dns-default-b57f6\" (UID: \"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593\") " pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.624925 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624896    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-crio-socket\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.625144 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624937    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-data-volume\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.625144 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.624945    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.628047 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.628028    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.628119 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.628051    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e82d0f20-1cb3-4f5f-a04e-3e10b5f76593-metrics-tls\") pod \"dns-default-b57f6\" (UID: \"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593\") " pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.628252 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.628231    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7d9e375-8217-427d-81d9-ae0aaa546025-cert\") pod \"ingress-canary-wjjwx\" (UID: \"a7d9e375-8217-427d-81d9-ae0aaa546025\") " pod="openshift-ingress-canary/ingress-canary-wjjwx"
Jul 13 21:59:25.633055 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.633034    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkxrt\" (UniqueName: \"kubernetes.io/projected/e82d0f20-1cb3-4f5f-a04e-3e10b5f76593-kube-api-access-mkxrt\") pod \"dns-default-b57f6\" (UID: \"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593\") " pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.633140 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.633063    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p46hn\" (UniqueName: \"kubernetes.io/projected/a7d9e375-8217-427d-81d9-ae0aaa546025-kube-api-access-p46hn\") pod \"ingress-canary-wjjwx\" (UID: \"a7d9e375-8217-427d-81d9-ae0aaa546025\") " pod="openshift-ingress-canary/ingress-canary-wjjwx"
Jul 13 21:59:25.633287 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.633269    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxmmj\" (UniqueName: \"kubernetes.io/projected/dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b-kube-api-access-lxmmj\") pod \"insights-runtime-extractor-jsl8d\" (UID: \"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b\") " pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.782580 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.782552    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:25.801677 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.801656    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wjjwx"
Jul 13 21:59:25.814851 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.814834    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jsl8d"
Jul 13 21:59:25.995054 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:25.995025    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wjjwx"]
Jul 13 21:59:26.002569 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.000236    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b57f6"]
Jul 13 21:59:26.002717 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.002663    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jsl8d"]
Jul 13 21:59:26.006256 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:59:26.006232    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode82d0f20_1cb3_4f5f_a04e_3e10b5f76593.slice/crio-dbda1b911dd4406dbd04c9e667d4332f0e406d4fab3c727515ff2b90314b42e6 WatchSource:0}: Error finding container dbda1b911dd4406dbd04c9e667d4332f0e406d4fab3c727515ff2b90314b42e6: Status 404 returned error can't find the container with id dbda1b911dd4406dbd04c9e667d4332f0e406d4fab3c727515ff2b90314b42e6
Jul 13 21:59:26.006912 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:59:26.006824    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc2f9a6c_cdf8_48f5_a66e_507c0dcbfe2b.slice/crio-b87e87e9af30dfcc6c851bfa775164d4f638697f6567a3309bd593fa29a33455 WatchSource:0}: Error finding container b87e87e9af30dfcc6c851bfa775164d4f638697f6567a3309bd593fa29a33455: Status 404 returned error can't find the container with id b87e87e9af30dfcc6c851bfa775164d4f638697f6567a3309bd593fa29a33455
Jul 13 21:59:26.660381 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.660350    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:26.660825 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.660806    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:26.661224 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.661205    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:26.666582 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.666003    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 21:59:26.666582 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.666084    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 21:59:26.666582 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.666004    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-djqhc\""
Jul 13 21:59:26.666582 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.666293    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 21:59:26.666582 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.666397    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vvmxw\""
Jul 13 21:59:26.666582 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.666457    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 21:59:26.949680 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.949595    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jsl8d" event={"ID":"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b","Type":"ContainerStarted","Data":"b87e87e9af30dfcc6c851bfa775164d4f638697f6567a3309bd593fa29a33455"}
Jul 13 21:59:26.951970 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.951926    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wjjwx" event={"ID":"a7d9e375-8217-427d-81d9-ae0aaa546025","Type":"ContainerStarted","Data":"56479d01ee49b0474c220042176b2cc2cdcc8154770808f1ccabc58677884616"}
Jul 13 21:59:26.954816 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:26.954762    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b57f6" event={"ID":"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593","Type":"ContainerStarted","Data":"dbda1b911dd4406dbd04c9e667d4332f0e406d4fab3c727515ff2b90314b42e6"}
Jul 13 21:59:31.057725 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.057488    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-c22nr"]
Jul 13 21:59:31.125342 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.125305    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-c22nr"]
Jul 13 21:59:31.125489 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.125421    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.128304 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.128280    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 13 21:59:31.128304 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.128282    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 13 21:59:31.128492 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.128343    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 13 21:59:31.128607 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.128590    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 13 21:59:31.129397 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.129375    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-zmkxs\""
Jul 13 21:59:31.129498 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.129481    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 13 21:59:31.164167 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.164139    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsr6v\" (UniqueName: \"kubernetes.io/projected/22d05f1c-bcb1-4a97-92cb-078a1025393d-kube-api-access-vsr6v\") pod \"prometheus-operator-64785b6675-c22nr\" (UID: \"22d05f1c-bcb1-4a97-92cb-078a1025393d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.164302 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.164188    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/22d05f1c-bcb1-4a97-92cb-078a1025393d-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-c22nr\" (UID: \"22d05f1c-bcb1-4a97-92cb-078a1025393d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.164302 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.164264    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22d05f1c-bcb1-4a97-92cb-078a1025393d-metrics-client-ca\") pod \"prometheus-operator-64785b6675-c22nr\" (UID: \"22d05f1c-bcb1-4a97-92cb-078a1025393d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.164395 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.164303    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22d05f1c-bcb1-4a97-92cb-078a1025393d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-c22nr\" (UID: \"22d05f1c-bcb1-4a97-92cb-078a1025393d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.265098 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.265065    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsr6v\" (UniqueName: \"kubernetes.io/projected/22d05f1c-bcb1-4a97-92cb-078a1025393d-kube-api-access-vsr6v\") pod \"prometheus-operator-64785b6675-c22nr\" (UID: \"22d05f1c-bcb1-4a97-92cb-078a1025393d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.265098 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.265102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/22d05f1c-bcb1-4a97-92cb-078a1025393d-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-c22nr\" (UID: \"22d05f1c-bcb1-4a97-92cb-078a1025393d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.265319 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.265130    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22d05f1c-bcb1-4a97-92cb-078a1025393d-metrics-client-ca\") pod \"prometheus-operator-64785b6675-c22nr\" (UID: \"22d05f1c-bcb1-4a97-92cb-078a1025393d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.265319 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.265164    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22d05f1c-bcb1-4a97-92cb-078a1025393d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-c22nr\" (UID: \"22d05f1c-bcb1-4a97-92cb-078a1025393d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.265319 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:31.265262    2558 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 13 21:59:31.265460 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:31.265345    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22d05f1c-bcb1-4a97-92cb-078a1025393d-prometheus-operator-tls podName:22d05f1c-bcb1-4a97-92cb-078a1025393d nodeName:}" failed. No retries permitted until 2026-07-13 21:59:31.765323934 +0000 UTC m=+59.683300387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/22d05f1c-bcb1-4a97-92cb-078a1025393d-prometheus-operator-tls") pod "prometheus-operator-64785b6675-c22nr" (UID: "22d05f1c-bcb1-4a97-92cb-078a1025393d") : secret "prometheus-operator-tls" not found
Jul 13 21:59:31.266001 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.265951    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22d05f1c-bcb1-4a97-92cb-078a1025393d-metrics-client-ca\") pod \"prometheus-operator-64785b6675-c22nr\" (UID: \"22d05f1c-bcb1-4a97-92cb-078a1025393d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.268834 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.268808    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22d05f1c-bcb1-4a97-92cb-078a1025393d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-c22nr\" (UID: \"22d05f1c-bcb1-4a97-92cb-078a1025393d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.276497 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.276472    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsr6v\" (UniqueName: \"kubernetes.io/projected/22d05f1c-bcb1-4a97-92cb-078a1025393d-kube-api-access-vsr6v\") pod \"prometheus-operator-64785b6675-c22nr\" (UID: \"22d05f1c-bcb1-4a97-92cb-078a1025393d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.769110 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.769089    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/22d05f1c-bcb1-4a97-92cb-078a1025393d-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-c22nr\" (UID: \"22d05f1c-bcb1-4a97-92cb-078a1025393d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.771300 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.771278    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/22d05f1c-bcb1-4a97-92cb-078a1025393d-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-c22nr\" (UID: \"22d05f1c-bcb1-4a97-92cb-078a1025393d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:31.970266 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.970225    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jsl8d" event={"ID":"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b","Type":"ContainerStarted","Data":"0ea1be36dbd7be0b9fda9b763678d141a677f96a7be4b431cc7ba703845c6da4"}
Jul 13 21:59:31.971592 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.971559    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wjjwx" event={"ID":"a7d9e375-8217-427d-81d9-ae0aaa546025","Type":"ContainerStarted","Data":"bbb58d50e362cd88c8fe648a41cc54c0642e274476f0a65364f8f2569a7b439a"}
Jul 13 21:59:31.973458 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.973432    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b57f6" event={"ID":"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593","Type":"ContainerStarted","Data":"f4204a8cd791e9dee4d13c02cb1e0b6ec2ada010feb47cf6ff2031e7b88fc9dd"}
Jul 13 21:59:31.973595 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.973472    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b57f6" event={"ID":"e82d0f20-1cb3-4f5f-a04e-3e10b5f76593","Type":"ContainerStarted","Data":"0ddc36187e55f03c0dae71515cfb83f39081dde5a872139ee07bc5a33e204528"}
Jul 13 21:59:31.973595 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.973500    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:31.988397 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:31.988356    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wjjwx" podStartSLOduration=1.2348186700000001 podStartE2EDuration="6.988341106s" totalImagesPullingTime="5.753522436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:31.987534674 +0000 UTC m=+59.905511139" watchObservedRunningTime="2026-07-13 21:59:31.988341106 +0000 UTC m=+59.906317565"
Jul 13 21:59:32.003925 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:32.003882    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b57f6" podStartSLOduration=1.255131875 podStartE2EDuration="7.003868059s" totalImagesPullingTime="5.748736184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:32.00373947 +0000 UTC m=+59.921715935" watchObservedRunningTime="2026-07-13 21:59:32.003868059 +0000 UTC m=+59.921844523"
Jul 13 21:59:32.035994 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:32.035917    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr"
Jul 13 21:59:32.161157 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:32.161129    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-c22nr"]
Jul 13 21:59:32.163958 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:59:32.163932    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d05f1c_bcb1_4a97_92cb_078a1025393d.slice/crio-adf485a848ddb6ee69940b83122d865bdaa65f5087860de21e4169e0b919b18b WatchSource:0}: Error finding container adf485a848ddb6ee69940b83122d865bdaa65f5087860de21e4169e0b919b18b: Status 404 returned error can't find the container with id adf485a848ddb6ee69940b83122d865bdaa65f5087860de21e4169e0b919b18b
Jul 13 21:59:32.892318 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:32.892287    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x6k99"
Jul 13 21:59:32.979531 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:32.979455    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr" event={"ID":"22d05f1c-bcb1-4a97-92cb-078a1025393d","Type":"ContainerStarted","Data":"adf485a848ddb6ee69940b83122d865bdaa65f5087860de21e4169e0b919b18b"}
Jul 13 21:59:32.982141 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:32.982100    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jsl8d" event={"ID":"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b","Type":"ContainerStarted","Data":"3e5b5f138b92423d167e98804eeb539bfe58f6ae462e11b88644be6a8675b283"}
Jul 13 21:59:33.986897 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:33.986862    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr" event={"ID":"22d05f1c-bcb1-4a97-92cb-078a1025393d","Type":"ContainerStarted","Data":"7dd752c8d32b236840a63929cce76443ad45a6158648f33d9cdd6b8639f1b607"}
Jul 13 21:59:33.987313 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:33.986927    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr" event={"ID":"22d05f1c-bcb1-4a97-92cb-078a1025393d","Type":"ContainerStarted","Data":"234d232cdca79af0d7e7c655ac3b66e3022e655545481c9d42b4d0b572cc7f4e"}
Jul 13 21:59:34.005748 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:34.005596    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-c22nr" podStartSLOduration=1.7948911010000002 podStartE2EDuration="3.005576801s" totalImagesPullingTime="1.2106857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:34.00499852 +0000 UTC m=+61.922974997" watchObservedRunningTime="2026-07-13 21:59:34.005576801 +0000 UTC m=+61.923553262"
Jul 13 21:59:35.421867 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.421786    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"]
Jul 13 21:59:35.424041 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.424021    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:35.427644 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.427611    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 13 21:59:35.427761 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.427661    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 13 21:59:35.427880 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.427868    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-k2v5s\""
Jul 13 21:59:35.440092 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.440067    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"]
Jul 13 21:59:35.440756 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.440739    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"]
Jul 13 21:59:35.442661 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.442647    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.445208 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.445191    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 13 21:59:35.445574 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.445554    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 13 21:59:35.445669 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.445593    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 13 21:59:35.445739 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.445707    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7cf8f\""
Jul 13 21:59:35.455347 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.455326    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hvp9r"]
Jul 13 21:59:35.457466 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.457448    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.459171 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.459036    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"]
Jul 13 21:59:35.460332 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.460312    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 13 21:59:35.460451 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.460339    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 13 21:59:35.460451 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.460411    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lqdmm\""
Jul 13 21:59:35.460602 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.460476    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 13 21:59:35.498334 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498311    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/306c0019-4dd3-4c1e-885b-77dba7eb9c26-sys\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.498416 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498343    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fd51438-aa61-4d28-807a-6e48e4f2debb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wjh7h\" (UID: \"1fd51438-aa61-4d28-807a-6e48e4f2debb\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:35.498416 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498368    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c3b45145-5be1-4091-9890-fd896bdc1b55-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.498495 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498415    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h5s5\" (UniqueName: \"kubernetes.io/projected/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-api-access-5h5s5\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.498495 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498451    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1fd51438-aa61-4d28-807a-6e48e4f2debb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wjh7h\" (UID: \"1fd51438-aa61-4d28-807a-6e48e4f2debb\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:35.498495 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498470    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.498610 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498502    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n8h6\" (UniqueName: \"kubernetes.io/projected/1fd51438-aa61-4d28-807a-6e48e4f2debb-kube-api-access-2n8h6\") pod \"openshift-state-metrics-84bd68b896-wjh7h\" (UID: \"1fd51438-aa61-4d28-807a-6e48e4f2debb\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:35.498610 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498518    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-textfile\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.498610 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498557    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.498610 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498578    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.498610 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498600    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3b45145-5be1-4091-9890-fd896bdc1b55-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.498754 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498617    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-wtmp\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.498754 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498656    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-tls\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.498754 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/306c0019-4dd3-4c1e-885b-77dba7eb9c26-root\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.498754 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498720    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/306c0019-4dd3-4c1e-885b-77dba7eb9c26-metrics-client-ca\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.498754 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498737    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.498935 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498769    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m2rr\" (UniqueName: \"kubernetes.io/projected/306c0019-4dd3-4c1e-885b-77dba7eb9c26-kube-api-access-5m2rr\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.498935 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498795    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-accelerators-collector-config\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.498935 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.498818    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fd51438-aa61-4d28-807a-6e48e4f2debb-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wjh7h\" (UID: \"1fd51438-aa61-4d28-807a-6e48e4f2debb\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:35.599809 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.599785    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n8h6\" (UniqueName: \"kubernetes.io/projected/1fd51438-aa61-4d28-807a-6e48e4f2debb-kube-api-access-2n8h6\") pod \"openshift-state-metrics-84bd68b896-wjh7h\" (UID: \"1fd51438-aa61-4d28-807a-6e48e4f2debb\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:35.599809 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.599810    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-textfile\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.599941 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.599829    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.599941 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.599846    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.599941 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.599893    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3b45145-5be1-4091-9890-fd896bdc1b55-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.599941 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.599922    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-wtmp\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.600111 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:35.599939    2558 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 13 21:59:35.600111 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.599941    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-tls\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.600111 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:35.600011    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 13 21:59:35.600111 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:35.600026    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-state-metrics-tls podName:c3b45145-5be1-4091-9890-fd896bdc1b55 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:36.099983989 +0000 UTC m=+64.017960437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-5xtg7" (UID: "c3b45145-5be1-4091-9890-fd896bdc1b55") : secret "kube-state-metrics-tls" not found
Jul 13 21:59:35.600111 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:35.600058    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-tls podName:306c0019-4dd3-4c1e-885b-77dba7eb9c26 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:36.100042867 +0000 UTC m=+64.018019309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-tls") pod "node-exporter-hvp9r" (UID: "306c0019-4dd3-4c1e-885b-77dba7eb9c26") : secret "node-exporter-tls" not found
Jul 13 21:59:35.600361 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600125    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/306c0019-4dd3-4c1e-885b-77dba7eb9c26-root\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.600361 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600127    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-wtmp\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.600361 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600165    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-textfile\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.600361 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600181    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/306c0019-4dd3-4c1e-885b-77dba7eb9c26-metrics-client-ca\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.600361 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600207    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.600361 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600226    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/306c0019-4dd3-4c1e-885b-77dba7eb9c26-root\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.600361 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600279    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m2rr\" (UniqueName: \"kubernetes.io/projected/306c0019-4dd3-4c1e-885b-77dba7eb9c26-kube-api-access-5m2rr\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.600361 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600304    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-accelerators-collector-config\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.600361 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600334    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fd51438-aa61-4d28-807a-6e48e4f2debb-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wjh7h\" (UID: \"1fd51438-aa61-4d28-807a-6e48e4f2debb\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:35.600361 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600365    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/306c0019-4dd3-4c1e-885b-77dba7eb9c26-sys\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.600850 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600396    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fd51438-aa61-4d28-807a-6e48e4f2debb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wjh7h\" (UID: \"1fd51438-aa61-4d28-807a-6e48e4f2debb\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:35.600850 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600429    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c3b45145-5be1-4091-9890-fd896bdc1b55-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.600850 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600456    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h5s5\" (UniqueName: \"kubernetes.io/projected/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-api-access-5h5s5\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.600850 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600481    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1fd51438-aa61-4d28-807a-6e48e4f2debb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wjh7h\" (UID: \"1fd51438-aa61-4d28-807a-6e48e4f2debb\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:35.600850 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600512    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.600850 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600731    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3b45145-5be1-4091-9890-fd896bdc1b55-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.600850 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600765    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.600850 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600798    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/306c0019-4dd3-4c1e-885b-77dba7eb9c26-metrics-client-ca\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.601181 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:35.600894    2558 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 13 21:59:35.601181 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600902    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-accelerators-collector-config\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.601181 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:35.600950    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fd51438-aa61-4d28-807a-6e48e4f2debb-openshift-state-metrics-tls podName:1fd51438-aa61-4d28-807a-6e48e4f2debb nodeName:}" failed. No retries permitted until 2026-07-13 21:59:36.100933539 +0000 UTC m=+64.018909994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1fd51438-aa61-4d28-807a-6e48e4f2debb-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-wjh7h" (UID: "1fd51438-aa61-4d28-807a-6e48e4f2debb") : secret "openshift-state-metrics-tls" not found
Jul 13 21:59:35.601181 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.600984    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/306c0019-4dd3-4c1e-885b-77dba7eb9c26-sys\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.601181 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.601168    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c3b45145-5be1-4091-9890-fd896bdc1b55-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.601436 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.601413    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fd51438-aa61-4d28-807a-6e48e4f2debb-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-wjh7h\" (UID: \"1fd51438-aa61-4d28-807a-6e48e4f2debb\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:35.602669 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.602650    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.602911 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.602892    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fd51438-aa61-4d28-807a-6e48e4f2debb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-wjh7h\" (UID: \"1fd51438-aa61-4d28-807a-6e48e4f2debb\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:35.602976 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.602930    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.608719 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.608697    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m2rr\" (UniqueName: \"kubernetes.io/projected/306c0019-4dd3-4c1e-885b-77dba7eb9c26-kube-api-access-5m2rr\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:35.608903 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.608879    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n8h6\" (UniqueName: \"kubernetes.io/projected/1fd51438-aa61-4d28-807a-6e48e4f2debb-kube-api-access-2n8h6\") pod \"openshift-state-metrics-84bd68b896-wjh7h\" (UID: \"1fd51438-aa61-4d28-807a-6e48e4f2debb\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:35.610980 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.610960    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h5s5\" (UniqueName: \"kubernetes.io/projected/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-api-access-5h5s5\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:35.995245 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:35.995216    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jsl8d" event={"ID":"dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b","Type":"ContainerStarted","Data":"53906e016ed6d12695a4f50bcdc3236b57dad94fb6e80a21fdfd25374866e166"}
Jul 13 21:59:36.016948 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.016906    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jsl8d" podStartSLOduration=2.133052047 podStartE2EDuration="11.016891665s" totalImagesPullingTime="8.883839618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:59:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:36.015670615 +0000 UTC m=+63.933647090" watchObservedRunningTime="2026-07-13 21:59:36.016891665 +0000 UTC m=+63.934868107"
Jul 13 21:59:36.104860 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.104834    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1fd51438-aa61-4d28-807a-6e48e4f2debb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wjh7h\" (UID: \"1fd51438-aa61-4d28-807a-6e48e4f2debb\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:36.104975 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.104901    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:36.104975 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.104938    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-tls\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:36.105080 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:36.105030    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 13 21:59:36.105165 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:36.105152    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-tls podName:306c0019-4dd3-4c1e-885b-77dba7eb9c26 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:37.105134613 +0000 UTC m=+65.023111079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-tls") pod "node-exporter-hvp9r" (UID: "306c0019-4dd3-4c1e-885b-77dba7eb9c26") : secret "node-exporter-tls" not found
Jul 13 21:59:36.107148 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.107128    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3b45145-5be1-4091-9890-fd896bdc1b55-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-5xtg7\" (UID: \"c3b45145-5be1-4091-9890-fd896bdc1b55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:36.107210 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.107178    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1fd51438-aa61-4d28-807a-6e48e4f2debb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-wjh7h\" (UID: \"1fd51438-aa61-4d28-807a-6e48e4f2debb\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:36.332486 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.332423    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"
Jul 13 21:59:36.351329 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.351304    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"
Jul 13 21:59:36.470739 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.470708    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h"]
Jul 13 21:59:36.487944 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.487921    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-5xtg7"]
Jul 13 21:59:36.490532 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:59:36.490510    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3b45145_5be1_4091_9890_fd896bdc1b55.slice/crio-89a09419532f6da75d9cad35ba8973a8fcb4bb98f080265f9f5852aaaf35d4fd WatchSource:0}: Error finding container 89a09419532f6da75d9cad35ba8973a8fcb4bb98f080265f9f5852aaaf35d4fd: Status 404 returned error can't find the container with id 89a09419532f6da75d9cad35ba8973a8fcb4bb98f080265f9f5852aaaf35d4fd
Jul 13 21:59:36.503594 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.503571    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:59:36.509127 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.509108    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.512299 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.512277    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 13 21:59:36.512418 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.512290    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 13 21:59:36.512773 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.512758    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 13 21:59:36.512845 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.512792    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 13 21:59:36.513047 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.513032    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 13 21:59:36.513096 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.513065    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 13 21:59:36.513148 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.513097    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xbw5x\""
Jul 13 21:59:36.513438 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.513422    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 13 21:59:36.513574 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.513456    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 13 21:59:36.513574 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.513476    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 13 21:59:36.535340 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.535290    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:59:36.609141 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.609116    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.609237 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.609148    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.609237 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.609167    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.609237 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.609209    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.609237 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.609234    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.609370 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.609265    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-config-out\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.609370 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.609305    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.609370 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.609348    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-config-volume\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.609452 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.609383    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v45d2\" (UniqueName: \"kubernetes.io/projected/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-kube-api-access-v45d2\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.609490 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.609473    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.609524 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.609497    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.609576 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.609521    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.609576 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.609551    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-web-config\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.709998 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.709974    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.709998 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.709996    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.710176 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.710017    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.710176 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.710032    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-web-config\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.710176 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.710056    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.710176 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:36.710137    2558 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 13 21:59:36.710368 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.710186    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.710368 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:36.710201    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-main-tls podName:4f24bf8c-c597-4c3f-87fd-6b321dd0d900 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:37.210180747 +0000 UTC m=+65.128157205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "4f24bf8c-c597-4c3f-87fd-6b321dd0d900") : secret "alertmanager-main-tls" not found
Jul 13 21:59:36.710368 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.710232    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.710368 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:36.710286    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-alertmanager-trusted-ca-bundle podName:4f24bf8c-c597-4c3f-87fd-6b321dd0d900 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:37.210272634 +0000 UTC m=+65.128249101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "4f24bf8c-c597-4c3f-87fd-6b321dd0d900") : configmap references non-existent config key: ca-bundle.crt
Jul 13 21:59:36.710368 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.710302    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.710368 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.710342    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.710695 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.710383    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-config-out\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.710695 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.710427    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.710695 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.710455    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-config-volume\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.710695 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.710482    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v45d2\" (UniqueName: \"kubernetes.io/projected/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-kube-api-access-v45d2\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.710974 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.710953    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.711231 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.711191    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.713154 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.713109    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.713253 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.713167    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.713253 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.713189    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-web-config\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.713640 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.713603    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.713823 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.713763    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.714582 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.714525    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.715451 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.715425    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-config-out\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.715717 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.715702    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-config-volume\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:36.720256 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:36.720235    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v45d2\" (UniqueName: \"kubernetes.io/projected/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-kube-api-access-v45d2\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:37.000504 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:37.000469    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h" event={"ID":"1fd51438-aa61-4d28-807a-6e48e4f2debb","Type":"ContainerStarted","Data":"c01ae917b64f96139c05f75635c91b4211a47f0657ac053cea5d6dc9a09ceb5e"}
Jul 13 21:59:37.000685 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:37.000533    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h" event={"ID":"1fd51438-aa61-4d28-807a-6e48e4f2debb","Type":"ContainerStarted","Data":"67a622e70ca9e950272fb5c5b27475e86b4807498701879ff4258c274ba8536b"}
Jul 13 21:59:37.000685 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:37.000576    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h" event={"ID":"1fd51438-aa61-4d28-807a-6e48e4f2debb","Type":"ContainerStarted","Data":"c15d184344341660ca2bf77bd007e62989fbc2cd526f3b8ff4ed522655a6f82e"}
Jul 13 21:59:37.001620 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:37.001595    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7" event={"ID":"c3b45145-5be1-4091-9890-fd896bdc1b55","Type":"ContainerStarted","Data":"89a09419532f6da75d9cad35ba8973a8fcb4bb98f080265f9f5852aaaf35d4fd"}
Jul 13 21:59:37.113191 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:37.113136    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-tls\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:37.115802 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:37.115777    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/306c0019-4dd3-4c1e-885b-77dba7eb9c26-node-exporter-tls\") pod \"node-exporter-hvp9r\" (UID: \"306c0019-4dd3-4c1e-885b-77dba7eb9c26\") " pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:37.214799 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:37.214764    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:37.214957 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:37.214828    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:37.215792 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:37.215769    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:37.217430 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:37.217407    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4f24bf8c-c597-4c3f-87fd-6b321dd0d900-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4f24bf8c-c597-4c3f-87fd-6b321dd0d900\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:37.266882 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:37.266808    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hvp9r"
Jul 13 21:59:37.279710 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:59:37.279669    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306c0019_4dd3_4c1e_885b_77dba7eb9c26.slice/crio-5d5ab007f4bd3bcfb09dddf516f8a7736ac88cd0a79c48104f9bb57a191ab170 WatchSource:0}: Error finding container 5d5ab007f4bd3bcfb09dddf516f8a7736ac88cd0a79c48104f9bb57a191ab170: Status 404 returned error can't find the container with id 5d5ab007f4bd3bcfb09dddf516f8a7736ac88cd0a79c48104f9bb57a191ab170
Jul 13 21:59:37.423114 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:37.423083    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:59:37.715313 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:37.715286    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:59:37.909055 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:59:37.908982    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f24bf8c_c597_4c3f_87fd_6b321dd0d900.slice/crio-84c30a9d613fec267268e82af61f137a86b779494773a56c6e6ca0d3d4b58336 WatchSource:0}: Error finding container 84c30a9d613fec267268e82af61f137a86b779494773a56c6e6ca0d3d4b58336: Status 404 returned error can't find the container with id 84c30a9d613fec267268e82af61f137a86b779494773a56c6e6ca0d3d4b58336
Jul 13 21:59:38.008928 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.008841    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7" event={"ID":"c3b45145-5be1-4091-9890-fd896bdc1b55","Type":"ContainerStarted","Data":"02671e87da16e7293783f409bf4fa72bd4ccffff30c8ccf834b6e72394570bab"}
Jul 13 21:59:38.010503 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.010464    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f24bf8c-c597-4c3f-87fd-6b321dd0d900","Type":"ContainerStarted","Data":"84c30a9d613fec267268e82af61f137a86b779494773a56c6e6ca0d3d4b58336"}
Jul 13 21:59:38.011802 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.011778    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hvp9r" event={"ID":"306c0019-4dd3-4c1e-885b-77dba7eb9c26","Type":"ContainerStarted","Data":"5d5ab007f4bd3bcfb09dddf516f8a7736ac88cd0a79c48104f9bb57a191ab170"}
Jul 13 21:59:38.013990 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.013939    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h" event={"ID":"1fd51438-aa61-4d28-807a-6e48e4f2debb","Type":"ContainerStarted","Data":"1da124ab7a3c07b354195c19d3f2dd46070fba849d5a3cdd617ccba59512cb26"}
Jul 13 21:59:38.068096 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.067814    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-wjh7h" podStartSLOduration=2.090691156 podStartE2EDuration="3.067779158s" totalImagesPullingTime="977.088002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:38.066901305 +0000 UTC m=+65.984877772" watchObservedRunningTime="2026-07-13 21:59:38.067779158 +0000 UTC m=+65.985755624"
Jul 13 21:59:38.427709 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.427673    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:38.427873 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.427773    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs\") pod \"network-metrics-daemon-bx7hm\" (UID: \"b485735e-e460-453b-8503-c034e753bf78\") " pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:38.430482 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.430458    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 21:59:38.430626 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.430531    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 21:59:38.441910 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.441853    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/191c77b5-4225-4b5d-9402-ccdc1765dde8-original-pull-secret\") pod \"global-pull-secret-syncer-7n4kx\" (UID: \"191c77b5-4225-4b5d-9402-ccdc1765dde8\") " pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:38.443981 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.443916    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b485735e-e460-453b-8503-c034e753bf78-metrics-certs\") pod \"network-metrics-daemon-bx7hm\" (UID: \"b485735e-e460-453b-8503-c034e753bf78\") " pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:38.529150 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.529125    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ccsd\" (UniqueName: \"kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd\") pod \"network-check-target-csbjt\" (UID: \"2ad8149f-fe74-4aa2-8f43-bef582499a31\") " pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:38.535001 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.534982    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 21:59:38.552224 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.552205    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 21:59:38.562952 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.562932    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ccsd\" (UniqueName: \"kubernetes.io/projected/2ad8149f-fe74-4aa2-8f43-bef582499a31-kube-api-access-6ccsd\") pod \"network-check-target-csbjt\" (UID: \"2ad8149f-fe74-4aa2-8f43-bef582499a31\") " pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:38.686477 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.686421    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-djqhc\""
Jul 13 21:59:38.688435 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.688413    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:38.689664 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.689621    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7n4kx"
Jul 13 21:59:38.708996 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.708941    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vvmxw\""
Jul 13 21:59:38.717206 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.717114    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx7hm"
Jul 13 21:59:38.848997 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.848908    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7n4kx"]
Jul 13 21:59:38.856208 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:59:38.856169    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191c77b5_4225_4b5d_9402_ccdc1765dde8.slice/crio-fbf0ec658e5787a582c6ecf3b21868ca3930f9a5178e7144790757e209f10f74 WatchSource:0}: Error finding container fbf0ec658e5787a582c6ecf3b21868ca3930f9a5178e7144790757e209f10f74: Status 404 returned error can't find the container with id fbf0ec658e5787a582c6ecf3b21868ca3930f9a5178e7144790757e209f10f74
Jul 13 21:59:38.872168 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.872145    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-csbjt"]
Jul 13 21:59:38.875484 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:59:38.875455    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ad8149f_fe74_4aa2_8f43_bef582499a31.slice/crio-fbef8f45e1ae0d697af806eb59a5a7604b4a260fe92cafdb3b3b90abd95ebf06 WatchSource:0}: Error finding container fbef8f45e1ae0d697af806eb59a5a7604b4a260fe92cafdb3b3b90abd95ebf06: Status 404 returned error can't find the container with id fbef8f45e1ae0d697af806eb59a5a7604b4a260fe92cafdb3b3b90abd95ebf06
Jul 13 21:59:38.890305 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:38.890278    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bx7hm"]
Jul 13 21:59:38.893205 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:59:38.893185    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb485735e_e460_453b_8503_c034e753bf78.slice/crio-a2161a2ce9e4c54ec693498bf5cf6122269c0d060e06a29fffd11aec479dafbe WatchSource:0}: Error finding container a2161a2ce9e4c54ec693498bf5cf6122269c0d060e06a29fffd11aec479dafbe: Status 404 returned error can't find the container with id a2161a2ce9e4c54ec693498bf5cf6122269c0d060e06a29fffd11aec479dafbe
Jul 13 21:59:39.019016 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:39.018976    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-csbjt" event={"ID":"2ad8149f-fe74-4aa2-8f43-bef582499a31","Type":"ContainerStarted","Data":"fbef8f45e1ae0d697af806eb59a5a7604b4a260fe92cafdb3b3b90abd95ebf06"}
Jul 13 21:59:39.021510 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:39.021478    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7" event={"ID":"c3b45145-5be1-4091-9890-fd896bdc1b55","Type":"ContainerStarted","Data":"df14d12849169271c6f949d3ea91ed978497003ce3d9499a2977f69df5114fe2"}
Jul 13 21:59:39.021647 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:39.021534    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7" event={"ID":"c3b45145-5be1-4091-9890-fd896bdc1b55","Type":"ContainerStarted","Data":"17dc72cf0e0d99005ed40def9a9991bb8f28ccf1a01bd7361e9a0b2edea34e82"}
Jul 13 21:59:39.023316 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:39.023292    2558 generic.go:356] "Generic (PLEG): container finished" podID="306c0019-4dd3-4c1e-885b-77dba7eb9c26" containerID="e24889ba6ee4ceffec2d6e7efe9c8b61c3c62039fb2f556130e13378fe7d6ba8" exitCode=0
Jul 13 21:59:39.023420 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:39.023385    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hvp9r" event={"ID":"306c0019-4dd3-4c1e-885b-77dba7eb9c26","Type":"ContainerDied","Data":"e24889ba6ee4ceffec2d6e7efe9c8b61c3c62039fb2f556130e13378fe7d6ba8"}
Jul 13 21:59:39.024755 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:39.024733    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bx7hm" event={"ID":"b485735e-e460-453b-8503-c034e753bf78","Type":"ContainerStarted","Data":"a2161a2ce9e4c54ec693498bf5cf6122269c0d060e06a29fffd11aec479dafbe"}
Jul 13 21:59:39.025981 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:39.025957    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7n4kx" event={"ID":"191c77b5-4225-4b5d-9402-ccdc1765dde8","Type":"ContainerStarted","Data":"fbf0ec658e5787a582c6ecf3b21868ca3930f9a5178e7144790757e209f10f74"}
Jul 13 21:59:39.044880 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:39.044830    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-5xtg7" podStartSLOduration=2.602042467 podStartE2EDuration="4.044816628s" totalImagesPullingTime="1.442774161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:39.04308221 +0000 UTC m=+66.961058693" watchObservedRunningTime="2026-07-13 21:59:39.044816628 +0000 UTC m=+66.962793092"
Jul 13 21:59:40.033472 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.033294    2558 generic.go:356] "Generic (PLEG): container finished" podID="4f24bf8c-c597-4c3f-87fd-6b321dd0d900" containerID="2dd9ecf0ba8400063d52f01b0139c04f33f68a6bd9e43f1bc5e9ce079c7f70ee" exitCode=0
Jul 13 21:59:40.033472 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.033384    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f24bf8c-c597-4c3f-87fd-6b321dd0d900","Type":"ContainerDied","Data":"2dd9ecf0ba8400063d52f01b0139c04f33f68a6bd9e43f1bc5e9ce079c7f70ee"}
Jul 13 21:59:40.038610 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.038213    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hvp9r" event={"ID":"306c0019-4dd3-4c1e-885b-77dba7eb9c26","Type":"ContainerStarted","Data":"a54c70200770a25970c9adf4ea1138d013f0ef92fe9c4b9c6fd4b6d5c2993b3f"}
Jul 13 21:59:40.038610 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.038264    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hvp9r" event={"ID":"306c0019-4dd3-4c1e-885b-77dba7eb9c26","Type":"ContainerStarted","Data":"b81f9f6b2fe05895537e3785c7fc93675239b945899e5267c8158ea205e15119"}
Jul 13 21:59:40.041487 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.041450    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bx7hm" event={"ID":"b485735e-e460-453b-8503-c034e753bf78","Type":"ContainerStarted","Data":"806098e9448768f26fcdc3aefabd333f488bb203dbec248213512f88f542ac27"}
Jul 13 21:59:40.272125 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.272064    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hvp9r" podStartSLOduration=3.990397566 podStartE2EDuration="5.272044681s" totalImagesPullingTime="1.217986792s" totalInitContainerRuntime="63.660323ms" isStatefulPod=true podCreationTimestamp="2026-07-13 21:59:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:40.123337581 +0000 UTC m=+68.041314046" watchObservedRunningTime="2026-07-13 21:59:40.272044681 +0000 UTC m=+68.190021152"
Jul 13 21:59:40.272314 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.272259    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"]
Jul 13 21:59:40.274769 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.274745    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"
Jul 13 21:59:40.278286 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.278260    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-w2sn6\""
Jul 13 21:59:40.278849 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.278672    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 13 21:59:40.286794 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.286737    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"]
Jul 13 21:59:40.346108 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.346081    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvqhr\" (UniqueName: \"kubernetes.io/projected/b663c68d-d2a4-4900-812e-a3242a12eb93-kube-api-access-lvqhr\") pod \"monitoring-plugin-5f68df7f58-lwxhg\" (UID: \"b663c68d-d2a4-4900-812e-a3242a12eb93\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"
Jul 13 21:59:40.346283 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.346122    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b663c68d-d2a4-4900-812e-a3242a12eb93-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-lwxhg\" (UID: \"b663c68d-d2a4-4900-812e-a3242a12eb93\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"
Jul 13 21:59:40.447203 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.447041    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvqhr\" (UniqueName: \"kubernetes.io/projected/b663c68d-d2a4-4900-812e-a3242a12eb93-kube-api-access-lvqhr\") pod \"monitoring-plugin-5f68df7f58-lwxhg\" (UID: \"b663c68d-d2a4-4900-812e-a3242a12eb93\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"
Jul 13 21:59:40.447203 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.447089    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b663c68d-d2a4-4900-812e-a3242a12eb93-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-lwxhg\" (UID: \"b663c68d-d2a4-4900-812e-a3242a12eb93\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"
Jul 13 21:59:40.447422 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:40.447213    2558 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 13 21:59:40.447422 ip-10-0-143-101 kubenswrapper[2558]: E0713 21:59:40.447278    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b663c68d-d2a4-4900-812e-a3242a12eb93-monitoring-plugin-cert podName:b663c68d-d2a4-4900-812e-a3242a12eb93 nodeName:}" failed. No retries permitted until 2026-07-13 21:59:40.947256074 +0000 UTC m=+68.865232518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b663c68d-d2a4-4900-812e-a3242a12eb93-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-lwxhg" (UID: "b663c68d-d2a4-4900-812e-a3242a12eb93") : secret "monitoring-plugin-cert" not found
Jul 13 21:59:40.459170 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.459117    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvqhr\" (UniqueName: \"kubernetes.io/projected/b663c68d-d2a4-4900-812e-a3242a12eb93-kube-api-access-lvqhr\") pod \"monitoring-plugin-5f68df7f58-lwxhg\" (UID: \"b663c68d-d2a4-4900-812e-a3242a12eb93\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"
Jul 13 21:59:40.952402 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.952372    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b663c68d-d2a4-4900-812e-a3242a12eb93-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-lwxhg\" (UID: \"b663c68d-d2a4-4900-812e-a3242a12eb93\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"
Jul 13 21:59:40.955112 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:40.955083    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b663c68d-d2a4-4900-812e-a3242a12eb93-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-lwxhg\" (UID: \"b663c68d-d2a4-4900-812e-a3242a12eb93\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"
Jul 13 21:59:41.051412 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.051289    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bx7hm" event={"ID":"b485735e-e460-453b-8503-c034e753bf78","Type":"ContainerStarted","Data":"04c755fa5ffa5ca6fc5556d4fbf16d6c1643bd29cfabd0498f4b7def56664671"}
Jul 13 21:59:41.072016 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.071958    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bx7hm" podStartSLOduration=67.056452385 podStartE2EDuration="1m8.071938646s" totalImagesPullingTime="1.015486261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:41.070829225 +0000 UTC m=+68.988805691" watchObservedRunningTime="2026-07-13 21:59:41.071938646 +0000 UTC m=+68.989915123"
Jul 13 21:59:41.189476 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.189449    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"
Jul 13 21:59:41.817049 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.816772    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 21:59:41.825622 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.824381    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.827184 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.827162    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 13 21:59:41.827285 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.827200    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 13 21:59:41.827502 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.827482    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 13 21:59:41.828023 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.827868    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 13 21:59:41.828519 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.828225    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 13 21:59:41.830753 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.830630    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 13 21:59:41.831134 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.831114    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1bp28n1v5fg1\""
Jul 13 21:59:41.831218 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.831143    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 13 21:59:41.831218 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.831191    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 13 21:59:41.831320 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.831248    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 13 21:59:41.831320 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.831313    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 13 21:59:41.831430 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.831119    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 13 21:59:41.831430 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.831356    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 13 21:59:41.831621 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.831582    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 13 21:59:41.831893 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.831875    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 13 21:59:41.831977 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.831966    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9lc6x\""
Jul 13 21:59:41.833946 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.833899    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 13 21:59:41.849652 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.849618    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 21:59:41.860806 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.860753    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.860806 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.860795    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.860982 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.860827    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.860982 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.860864    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.860982 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.860891    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.860982 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.860912    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-web-config\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.860982 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.860939    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.860982 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.860967    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.861186 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.860999    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.861186 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.861022    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.861186 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.861053    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-config\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.861186 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.861075    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhk6f\" (UniqueName: \"kubernetes.io/projected/5530f91e-4527-467a-94c4-7c14bd6efe21-kube-api-access-qhk6f\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.861186 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.861104    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5530f91e-4527-467a-94c4-7c14bd6efe21-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.861186 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.861128    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.861186 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.861159    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5530f91e-4527-467a-94c4-7c14bd6efe21-config-out\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.861473 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.861188    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.861473 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.861216    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.861473 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.861248    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.861473 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.861272    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.861473 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.861294    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962130 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962100    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-config\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962312 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962136    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhk6f\" (UniqueName: \"kubernetes.io/projected/5530f91e-4527-467a-94c4-7c14bd6efe21-kube-api-access-qhk6f\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962312 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962298    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5530f91e-4527-467a-94c4-7c14bd6efe21-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962435 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962339    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962435 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962379    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5530f91e-4527-467a-94c4-7c14bd6efe21-config-out\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962435 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962409    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962435 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962430    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962672 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962454    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962672 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962470    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962672 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962486    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962672 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962672 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962530    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962672 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962574    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962672 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962602    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.962672 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.962618    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.963671 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.963642    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.963793 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.963707    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.963862 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.963805    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.967355 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.963918    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.967355 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.964160    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.967355 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.964299    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-web-config\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.967355 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.964340    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.967355 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.964365    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.967355 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.964393    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.967355 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.964411    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.967355 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.965037    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.967355 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.965442    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.967355 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.966104    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5530f91e-4527-467a-94c4-7c14bd6efe21-config-out\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.967355 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.966298    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-config\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.967355 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.967203    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.968748 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.968221    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5530f91e-4527-467a-94c4-7c14bd6efe21-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.968748 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.968700    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.968899 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.968790    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5530f91e-4527-467a-94c4-7c14bd6efe21-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.970004 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.969964    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-web-config\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.970852 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.970835    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.974040 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.973985    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.974135 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.974105    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.974241 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.974141    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.975295 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.974926    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5530f91e-4527-467a-94c4-7c14bd6efe21-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.976730 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.976689    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhk6f\" (UniqueName: \"kubernetes.io/projected/5530f91e-4527-467a-94c4-7c14bd6efe21-kube-api-access-qhk6f\") pod \"prometheus-k8s-0\" (UID: \"5530f91e-4527-467a-94c4-7c14bd6efe21\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:41.985677 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:41.985660    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b57f6"
Jul 13 21:59:42.143146 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:42.143057    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:59:44.497569 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:44.496618    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 21:59:44.499211 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:59:44.499085    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5530f91e_4527_467a_94c4_7c14bd6efe21.slice/crio-0b0768314cf96ababc4fe38bf080063f05933475c08f139e242dfc5d600666df WatchSource:0}: Error finding container 0b0768314cf96ababc4fe38bf080063f05933475c08f139e242dfc5d600666df: Status 404 returned error can't find the container with id 0b0768314cf96ababc4fe38bf080063f05933475c08f139e242dfc5d600666df
Jul 13 21:59:44.499349 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:44.499325    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"]
Jul 13 21:59:44.501769 ip-10-0-143-101 kubenswrapper[2558]: W0713 21:59:44.501738    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb663c68d_d2a4_4900_812e_a3242a12eb93.slice/crio-50c2b9405d21315abf1b86fbef1b5d67a35b2c95226aad5ea18ba7dda488a1fe WatchSource:0}: Error finding container 50c2b9405d21315abf1b86fbef1b5d67a35b2c95226aad5ea18ba7dda488a1fe: Status 404 returned error can't find the container with id 50c2b9405d21315abf1b86fbef1b5d67a35b2c95226aad5ea18ba7dda488a1fe
Jul 13 21:59:45.071308 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.071267    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg" event={"ID":"b663c68d-d2a4-4900-812e-a3242a12eb93","Type":"ContainerStarted","Data":"50c2b9405d21315abf1b86fbef1b5d67a35b2c95226aad5ea18ba7dda488a1fe"}
Jul 13 21:59:45.073066 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.073035    2558 generic.go:356] "Generic (PLEG): container finished" podID="5530f91e-4527-467a-94c4-7c14bd6efe21" containerID="227869e7ea9aeaca7feef29184c31c635397c1c7331a158d82c4edf38a2d9ecb" exitCode=0
Jul 13 21:59:45.073196 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.073123    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5530f91e-4527-467a-94c4-7c14bd6efe21","Type":"ContainerDied","Data":"227869e7ea9aeaca7feef29184c31c635397c1c7331a158d82c4edf38a2d9ecb"}
Jul 13 21:59:45.073256 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.073241    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5530f91e-4527-467a-94c4-7c14bd6efe21","Type":"ContainerStarted","Data":"0b0768314cf96ababc4fe38bf080063f05933475c08f139e242dfc5d600666df"}
Jul 13 21:59:45.075563 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.075515    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7n4kx" event={"ID":"191c77b5-4225-4b5d-9402-ccdc1765dde8","Type":"ContainerStarted","Data":"62939218e4efcc5c94fd0868a500bed4724d4e207ea4b01c2a2c560b34482c5c"}
Jul 13 21:59:45.077793 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.077772    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-csbjt" event={"ID":"2ad8149f-fe74-4aa2-8f43-bef582499a31","Type":"ContainerStarted","Data":"88e6a9fdfe0fa946aaf8a97374501432532b55edb950863e3513d7688368fe2b"}
Jul 13 21:59:45.077909 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.077895    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 21:59:45.081267 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.081244    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f24bf8c-c597-4c3f-87fd-6b321dd0d900","Type":"ContainerStarted","Data":"333bb9b7b03518dbeeacab120f2cba347eeb814ea9e306cabeca5a7c9485a059"}
Jul 13 21:59:45.081378 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.081301    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f24bf8c-c597-4c3f-87fd-6b321dd0d900","Type":"ContainerStarted","Data":"7bb8917761c5548c4f7f111501419577ed1b82ba528b3c8037bf7881444aec6e"}
Jul 13 21:59:45.081378 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.081326    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f24bf8c-c597-4c3f-87fd-6b321dd0d900","Type":"ContainerStarted","Data":"570cb8f7aade7b4c51a9ea52b2b96c138e8c411e131c169c69c3354fab97ef50"}
Jul 13 21:59:45.081378 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.081359    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f24bf8c-c597-4c3f-87fd-6b321dd0d900","Type":"ContainerStarted","Data":"a48fe53eaf716a05c8dd6bdcd2152def245f5d2470899b6f3069d5e56d32dd18"}
Jul 13 21:59:45.081489 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.081388    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f24bf8c-c597-4c3f-87fd-6b321dd0d900","Type":"ContainerStarted","Data":"886b8eb876b142fa4c13886fc737806b737950574794eacdc9d17c62fcda66e2"}
Jul 13 21:59:45.122341 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.122296    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7n4kx" podStartSLOduration=66.653403207 podStartE2EDuration="1m12.12228035s" totalImagesPullingTime="5.468877143s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:45.121836452 +0000 UTC m=+73.039812917" watchObservedRunningTime="2026-07-13 21:59:45.12228035 +0000 UTC m=+73.040256794"
Jul 13 21:59:45.137403 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:45.137356    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-csbjt" podStartSLOduration=66.695223253 podStartE2EDuration="1m12.137338397s" totalImagesPullingTime="5.442115144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:45.135916757 +0000 UTC m=+73.053893228" watchObservedRunningTime="2026-07-13 21:59:45.137338397 +0000 UTC m=+73.055314863"
Jul 13 21:59:46.087661 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:46.087575    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg" event={"ID":"b663c68d-d2a4-4900-812e-a3242a12eb93","Type":"ContainerStarted","Data":"b56da1777b08512761f2b99b7c10b8b5e911cea90a8f02e3b01186fee422f69c"}
Jul 13 21:59:46.087661 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:46.087644    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"
Jul 13 21:59:46.091806 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:46.091771    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f24bf8c-c597-4c3f-87fd-6b321dd0d900","Type":"ContainerStarted","Data":"84984df982a6cd0022c9b1c677515a0d26470c23c340f25f233e7e71cdc45e29"}
Jul 13 21:59:46.093414 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:46.093395    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg"
Jul 13 21:59:46.102323 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:46.102259    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-lwxhg" podStartSLOduration=4.780342048 podStartE2EDuration="6.102242072s" totalImagesPullingTime="1.321900024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:46.10199197 +0000 UTC m=+74.019968435" watchObservedRunningTime="2026-07-13 21:59:46.102242072 +0000 UTC m=+74.020218535"
Jul 13 21:59:46.141609 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:46.141555    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.864148241 podStartE2EDuration="10.141523827s" totalImagesPullingTime="6.259723255s" totalInitContainerRuntime="17.652331ms" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:46.139623256 +0000 UTC m=+74.057599719" watchObservedRunningTime="2026-07-13 21:59:46.141523827 +0000 UTC m=+74.059500291"
Jul 13 21:59:49.109001 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:49.108971    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5530f91e-4527-467a-94c4-7c14bd6efe21","Type":"ContainerStarted","Data":"acc46611fa779aa106be99d486cce9ddb6144301e869bee4d4fe34c25c842252"}
Jul 13 21:59:49.109431 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:49.109035    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5530f91e-4527-467a-94c4-7c14bd6efe21","Type":"ContainerStarted","Data":"92fc82997fe9886f312f8e752b3b12d8485798827e56670f10d47bf2c420f63a"}
Jul 13 21:59:51.119341 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:51.119309    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5530f91e-4527-467a-94c4-7c14bd6efe21","Type":"ContainerStarted","Data":"b065937ef6717de32443f2bf2f094561dcd3c7564cf5d3ba78965bc22968f4f2"}
Jul 13 21:59:51.119764 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:51.119370    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5530f91e-4527-467a-94c4-7c14bd6efe21","Type":"ContainerStarted","Data":"8fbe1895854f953ac5effb9d6dbc58144fd948eaed39de3e2aaedf9ae2d2632c"}
Jul 13 21:59:51.119764 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:51.119394    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5530f91e-4527-467a-94c4-7c14bd6efe21","Type":"ContainerStarted","Data":"7ca12f6ce8a6e6381b68030cb37164301a71ae938724328b621eb822244a5a49"}
Jul 13 21:59:51.120116 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:51.119853    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5530f91e-4527-467a-94c4-7c14bd6efe21","Type":"ContainerStarted","Data":"3d584c394cce1e0a52bf12d8079eed06d6153ffcfabcd46c5ddb60598b311f05"}
Jul 13 21:59:51.162227 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:51.162152    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.894031394 podStartE2EDuration="10.162137964s" totalImagesPullingTime="5.241372162s" totalInitContainerRuntime="26.734408ms" isStatefulPod=false podCreationTimestamp="2026-07-13 21:59:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:59:51.159829676 +0000 UTC m=+79.077806141" watchObservedRunningTime="2026-07-13 21:59:51.162137964 +0000 UTC m=+79.080114428"
Jul 13 21:59:52.143282 ip-10-0-143-101 kubenswrapper[2558]: I0713 21:59:52.143253    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 22:00:16.095341 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:00:16.095231    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-csbjt"
Jul 13 22:00:42.143635 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:00:42.143596    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 22:00:42.163531 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:00:42.163509    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 22:00:42.332043 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:00:42.332018    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 22:02:33.492588 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.492556    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-g4969"]
Jul 13 22:02:33.494992 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.494969    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-g4969"
Jul 13 22:02:33.497418 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.497400    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 13 22:02:33.497845 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.497823    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 13 22:02:33.498427 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.498405    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-z74vx\""
Jul 13 22:02:33.506099 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.506078    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-g4969"]
Jul 13 22:02:33.545365 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.545347    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02b83ba4-5a22-40a9-8391-5cdf3294c142-bound-sa-token\") pod \"cert-manager-55b4d954d5-g4969\" (UID: \"02b83ba4-5a22-40a9-8391-5cdf3294c142\") " pod="cert-manager/cert-manager-55b4d954d5-g4969"
Jul 13 22:02:33.545477 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.545386    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mckwl\" (UniqueName: \"kubernetes.io/projected/02b83ba4-5a22-40a9-8391-5cdf3294c142-kube-api-access-mckwl\") pod \"cert-manager-55b4d954d5-g4969\" (UID: \"02b83ba4-5a22-40a9-8391-5cdf3294c142\") " pod="cert-manager/cert-manager-55b4d954d5-g4969"
Jul 13 22:02:33.646431 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.646401    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02b83ba4-5a22-40a9-8391-5cdf3294c142-bound-sa-token\") pod \"cert-manager-55b4d954d5-g4969\" (UID: \"02b83ba4-5a22-40a9-8391-5cdf3294c142\") " pod="cert-manager/cert-manager-55b4d954d5-g4969"
Jul 13 22:02:33.646572 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.646437    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mckwl\" (UniqueName: \"kubernetes.io/projected/02b83ba4-5a22-40a9-8391-5cdf3294c142-kube-api-access-mckwl\") pod \"cert-manager-55b4d954d5-g4969\" (UID: \"02b83ba4-5a22-40a9-8391-5cdf3294c142\") " pod="cert-manager/cert-manager-55b4d954d5-g4969"
Jul 13 22:02:33.654865 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.654837    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02b83ba4-5a22-40a9-8391-5cdf3294c142-bound-sa-token\") pod \"cert-manager-55b4d954d5-g4969\" (UID: \"02b83ba4-5a22-40a9-8391-5cdf3294c142\") " pod="cert-manager/cert-manager-55b4d954d5-g4969"
Jul 13 22:02:33.654983 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.654968    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mckwl\" (UniqueName: \"kubernetes.io/projected/02b83ba4-5a22-40a9-8391-5cdf3294c142-kube-api-access-mckwl\") pod \"cert-manager-55b4d954d5-g4969\" (UID: \"02b83ba4-5a22-40a9-8391-5cdf3294c142\") " pod="cert-manager/cert-manager-55b4d954d5-g4969"
Jul 13 22:02:33.825126 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.825066    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-g4969"
Jul 13 22:02:33.944806 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:33.944762    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-g4969"]
Jul 13 22:02:33.947399 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:02:33.947361    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02b83ba4_5a22_40a9_8391_5cdf3294c142.slice/crio-1792ca2f955fede05397aecc646b895a2d81d3f8ccfd397d03cccdf6315a6957 WatchSource:0}: Error finding container 1792ca2f955fede05397aecc646b895a2d81d3f8ccfd397d03cccdf6315a6957: Status 404 returned error can't find the container with id 1792ca2f955fede05397aecc646b895a2d81d3f8ccfd397d03cccdf6315a6957
Jul 13 22:02:34.706106 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:34.706068    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-g4969" event={"ID":"02b83ba4-5a22-40a9-8391-5cdf3294c142","Type":"ContainerStarted","Data":"1792ca2f955fede05397aecc646b895a2d81d3f8ccfd397d03cccdf6315a6957"}
Jul 13 22:02:37.718174 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:37.718143    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-g4969" event={"ID":"02b83ba4-5a22-40a9-8391-5cdf3294c142","Type":"ContainerStarted","Data":"659209701aae9a3e6cde7b60365bad88cadb0e7301c5b56a09732a532d03b275"}
Jul 13 22:02:37.738479 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:37.738426    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-g4969" podStartSLOduration=1.793644534 podStartE2EDuration="4.738409652s" totalImagesPullingTime="2.944765118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:02:37.736600353 +0000 UTC m=+245.654576817" watchObservedRunningTime="2026-07-13 22:02:37.738409652 +0000 UTC m=+245.656386117"
Jul 13 22:02:59.565775 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.563155    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"]
Jul 13 22:02:59.568145 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.568116    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.571557 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.571525    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 13 22:02:59.571819 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.571800    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 13 22:02:59.571997 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.571979    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 13 22:02:59.572062 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.572005    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8h4rb\""
Jul 13 22:02:59.572062 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.572032    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 13 22:02:59.582927 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.582904    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"]
Jul 13 22:02:59.748824 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.748796    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87452\" (UniqueName: \"kubernetes.io/projected/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-kube-api-access-87452\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.748943 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.748845    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-tmp\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.748943 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.748876    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-manifests\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.748943 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.748896    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-apiservice-cert\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.748943 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.748912    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-webhook-cert\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.849563 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.849488    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-webhook-cert\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.849563 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.849521    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87452\" (UniqueName: \"kubernetes.io/projected/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-kube-api-access-87452\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.849729 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.849593    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-tmp\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.849729 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.849624    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-manifests\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.849729 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.849674    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-apiservice-cert\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.850025 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.850009    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-manifests\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.851870 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.851850    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-tmp\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.851964 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.851948    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-apiservice-cert\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.852065 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.852049    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-webhook-cert\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.862773 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.862751    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87452\" (UniqueName: \"kubernetes.io/projected/8c7d2f0f-00fa-4176-a8ee-675c699ccf55-kube-api-access-87452\") pod \"opendatahub-operator-controller-manager-787f66975d-v5v7q\" (UID: \"8c7d2f0f-00fa-4176-a8ee-675c699ccf55\") " pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:02:59.878015 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:02:59.877993    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:03:00.009404 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:00.009374    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"]
Jul 13 22:03:00.013106 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:03:00.013082    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c7d2f0f_00fa_4176_a8ee_675c699ccf55.slice/crio-a1d45a3e8a4169c08b1dfbf146c76cf731c8145efc7e213d530e16cff419634b WatchSource:0}: Error finding container a1d45a3e8a4169c08b1dfbf146c76cf731c8145efc7e213d530e16cff419634b: Status 404 returned error can't find the container with id a1d45a3e8a4169c08b1dfbf146c76cf731c8145efc7e213d530e16cff419634b
Jul 13 22:03:00.800715 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:00.800628    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q" event={"ID":"8c7d2f0f-00fa-4176-a8ee-675c699ccf55","Type":"ContainerStarted","Data":"a1d45a3e8a4169c08b1dfbf146c76cf731c8145efc7e213d530e16cff419634b"}
Jul 13 22:03:03.814437 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:03.814402    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q" event={"ID":"8c7d2f0f-00fa-4176-a8ee-675c699ccf55","Type":"ContainerStarted","Data":"35ad430c879b4fe8163964c3f9051b047b0c7c14f9dea3d0496b2269ae4eea1d"}
Jul 13 22:03:04.823185 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:04.823151    2558 generic.go:356] "Generic (PLEG): container finished" podID="8c7d2f0f-00fa-4176-a8ee-675c699ccf55" containerID="35ad430c879b4fe8163964c3f9051b047b0c7c14f9dea3d0496b2269ae4eea1d" exitCode=0
Jul 13 22:03:04.823580 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:04.823193    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q" event={"ID":"8c7d2f0f-00fa-4176-a8ee-675c699ccf55","Type":"ContainerDied","Data":"35ad430c879b4fe8163964c3f9051b047b0c7c14f9dea3d0496b2269ae4eea1d"}
Jul 13 22:03:05.829042 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:05.829008    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q" event={"ID":"8c7d2f0f-00fa-4176-a8ee-675c699ccf55","Type":"ContainerStarted","Data":"8ae21bc1d613febe770e86c3b1799228a5a66f936a4e04cfae7bb4f7af079277"}
Jul 13 22:03:05.829387 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:05.829121    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:03:16.835470 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:16.835442    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q"
Jul 13 22:03:16.861422 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:16.861375    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-787f66975d-v5v7q" podStartSLOduration=13.628824775 podStartE2EDuration="17.861358172s" totalImagesPullingTime="3.255202925s" totalInitContainerRuntime="977.330472ms" isStatefulPod=false podCreationTimestamp="2026-07-13 22:02:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:03:05.853895243 +0000 UTC m=+273.771871710" watchObservedRunningTime="2026-07-13 22:03:16.861358172 +0000 UTC m=+284.779334637"
Jul 13 22:03:19.562652 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.562617    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"]
Jul 13 22:03:19.570778 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.570751    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:19.574324 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.574302    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 13 22:03:19.574467 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.574343    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 13 22:03:19.574467 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.574404    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 13 22:03:19.574467 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.574415    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 13 22:03:19.574467 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.574421    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-g2xcd\""
Jul 13 22:03:19.574467 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.574436    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 13 22:03:19.583215 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.583194    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"]
Jul 13 22:03:19.600634 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.600614    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2a3bd4ed-f867-4159-abfd-ff23ee03d0e1-metrics-cert\") pod \"lws-controller-manager-849844fc89-qzcbm\" (UID: \"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1\") " pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:19.600724 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.600644    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5mcm\" (UniqueName: \"kubernetes.io/projected/2a3bd4ed-f867-4159-abfd-ff23ee03d0e1-kube-api-access-n5mcm\") pod \"lws-controller-manager-849844fc89-qzcbm\" (UID: \"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1\") " pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:19.600724 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.600666    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2a3bd4ed-f867-4159-abfd-ff23ee03d0e1-manager-config\") pod \"lws-controller-manager-849844fc89-qzcbm\" (UID: \"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1\") " pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:19.600797 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.600759    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a3bd4ed-f867-4159-abfd-ff23ee03d0e1-cert\") pod \"lws-controller-manager-849844fc89-qzcbm\" (UID: \"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1\") " pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:19.702002 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.701978    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a3bd4ed-f867-4159-abfd-ff23ee03d0e1-cert\") pod \"lws-controller-manager-849844fc89-qzcbm\" (UID: \"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1\") " pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:19.702088 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.702021    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2a3bd4ed-f867-4159-abfd-ff23ee03d0e1-metrics-cert\") pod \"lws-controller-manager-849844fc89-qzcbm\" (UID: \"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1\") " pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:19.702088 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.702044    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5mcm\" (UniqueName: \"kubernetes.io/projected/2a3bd4ed-f867-4159-abfd-ff23ee03d0e1-kube-api-access-n5mcm\") pod \"lws-controller-manager-849844fc89-qzcbm\" (UID: \"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1\") " pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:19.702088 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.702070    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2a3bd4ed-f867-4159-abfd-ff23ee03d0e1-manager-config\") pod \"lws-controller-manager-849844fc89-qzcbm\" (UID: \"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1\") " pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:19.702749 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.702723    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2a3bd4ed-f867-4159-abfd-ff23ee03d0e1-manager-config\") pod \"lws-controller-manager-849844fc89-qzcbm\" (UID: \"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1\") " pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:19.704362 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.704343    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2a3bd4ed-f867-4159-abfd-ff23ee03d0e1-metrics-cert\") pod \"lws-controller-manager-849844fc89-qzcbm\" (UID: \"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1\") " pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:19.704435 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.704375    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a3bd4ed-f867-4159-abfd-ff23ee03d0e1-cert\") pod \"lws-controller-manager-849844fc89-qzcbm\" (UID: \"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1\") " pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:19.712341 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.712322    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5mcm\" (UniqueName: \"kubernetes.io/projected/2a3bd4ed-f867-4159-abfd-ff23ee03d0e1-kube-api-access-n5mcm\") pod \"lws-controller-manager-849844fc89-qzcbm\" (UID: \"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1\") " pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:19.879492 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:19.879417    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:20.017163 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:20.017087    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"]
Jul 13 22:03:20.019926 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:03:20.019897    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3bd4ed_f867_4159_abfd_ff23ee03d0e1.slice/crio-39904480fb54409820de4b8082f2c523a5e3eab7c6c9c50891fe69547d230b94 WatchSource:0}: Error finding container 39904480fb54409820de4b8082f2c523a5e3eab7c6c9c50891fe69547d230b94: Status 404 returned error can't find the container with id 39904480fb54409820de4b8082f2c523a5e3eab7c6c9c50891fe69547d230b94
Jul 13 22:03:20.886343 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:20.886303    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm" event={"ID":"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1","Type":"ContainerStarted","Data":"39904480fb54409820de4b8082f2c523a5e3eab7c6c9c50891fe69547d230b94"}
Jul 13 22:03:22.895335 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:22.895298    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm" event={"ID":"2a3bd4ed-f867-4159-abfd-ff23ee03d0e1","Type":"ContainerStarted","Data":"2935db1c39dfa10afd5a1e82cb40cab206d188e4b367d6056847f8030eec4cc7"}
Jul 13 22:03:22.895745 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:22.895390    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:22.919980 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:22.919934    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm" podStartSLOduration=1.623986744 podStartE2EDuration="3.919919728s" totalImagesPullingTime="2.295932984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:03:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:03:22.917962495 +0000 UTC m=+290.835938960" watchObservedRunningTime="2026-07-13 22:03:22.919919728 +0000 UTC m=+290.837896193"
Jul 13 22:03:29.667611 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.667582    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"]
Jul 13 22:03:29.670664 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.670647    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"
Jul 13 22:03:29.673948 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.673926    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 13 22:03:29.674158 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.674144    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 13 22:03:29.675038 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.675016    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 13 22:03:29.675152 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.675045    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9lvx4\""
Jul 13 22:03:29.683650 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.683635    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 13 22:03:29.702705 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.702681    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"]
Jul 13 22:03:29.783375 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.783350    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6e60169-9227-432c-ad99-13551dbc761a-tls-certs\") pod \"kube-auth-proxy-766bd547bb-vbd5d\" (UID: \"e6e60169-9227-432c-ad99-13551dbc761a\") " pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"
Jul 13 22:03:29.783484 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.783399    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6e60169-9227-432c-ad99-13551dbc761a-tmp\") pod \"kube-auth-proxy-766bd547bb-vbd5d\" (UID: \"e6e60169-9227-432c-ad99-13551dbc761a\") " pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"
Jul 13 22:03:29.783484 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.783477    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hjks\" (UniqueName: \"kubernetes.io/projected/e6e60169-9227-432c-ad99-13551dbc761a-kube-api-access-4hjks\") pod \"kube-auth-proxy-766bd547bb-vbd5d\" (UID: \"e6e60169-9227-432c-ad99-13551dbc761a\") " pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"
Jul 13 22:03:29.884243 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.884217    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6e60169-9227-432c-ad99-13551dbc761a-tls-certs\") pod \"kube-auth-proxy-766bd547bb-vbd5d\" (UID: \"e6e60169-9227-432c-ad99-13551dbc761a\") " pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"
Jul 13 22:03:29.884374 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.884280    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6e60169-9227-432c-ad99-13551dbc761a-tmp\") pod \"kube-auth-proxy-766bd547bb-vbd5d\" (UID: \"e6e60169-9227-432c-ad99-13551dbc761a\") " pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"
Jul 13 22:03:29.884374 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.884326    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hjks\" (UniqueName: \"kubernetes.io/projected/e6e60169-9227-432c-ad99-13551dbc761a-kube-api-access-4hjks\") pod \"kube-auth-proxy-766bd547bb-vbd5d\" (UID: \"e6e60169-9227-432c-ad99-13551dbc761a\") " pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"
Jul 13 22:03:29.886437 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.886414    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6e60169-9227-432c-ad99-13551dbc761a-tls-certs\") pod \"kube-auth-proxy-766bd547bb-vbd5d\" (UID: \"e6e60169-9227-432c-ad99-13551dbc761a\") " pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"
Jul 13 22:03:29.886437 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.886429    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6e60169-9227-432c-ad99-13551dbc761a-tmp\") pod \"kube-auth-proxy-766bd547bb-vbd5d\" (UID: \"e6e60169-9227-432c-ad99-13551dbc761a\") " pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"
Jul 13 22:03:29.899481 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.899460    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hjks\" (UniqueName: \"kubernetes.io/projected/e6e60169-9227-432c-ad99-13551dbc761a-kube-api-access-4hjks\") pod \"kube-auth-proxy-766bd547bb-vbd5d\" (UID: \"e6e60169-9227-432c-ad99-13551dbc761a\") " pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"
Jul 13 22:03:29.980012 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:29.979989    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"
Jul 13 22:03:30.117667 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:30.117639    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d"]
Jul 13 22:03:30.120890 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:03:30.120866    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e60169_9227_432c_ad99_13551dbc761a.slice/crio-56a6a17a4129051c462adad4087b4c2dd9c701e473eb5aa0ca85f876ceef3f1f WatchSource:0}: Error finding container 56a6a17a4129051c462adad4087b4c2dd9c701e473eb5aa0ca85f876ceef3f1f: Status 404 returned error can't find the container with id 56a6a17a4129051c462adad4087b4c2dd9c701e473eb5aa0ca85f876ceef3f1f
Jul 13 22:03:30.927381 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:30.927346    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d" event={"ID":"e6e60169-9227-432c-ad99-13551dbc761a","Type":"ContainerStarted","Data":"56a6a17a4129051c462adad4087b4c2dd9c701e473eb5aa0ca85f876ceef3f1f"}
Jul 13 22:03:32.538160 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:32.538128    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:03:32.538644 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:32.538411    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:03:32.543788 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:32.543768    2558 kubelet.go:1648] "Image garbage collection succeeded"
Jul 13 22:03:33.694751 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:33.694728    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 13 22:03:33.902364 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:33.902341    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-849844fc89-qzcbm"
Jul 13 22:03:33.940152 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:03:33.939946    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d" event={"ID":"e6e60169-9227-432c-ad99-13551dbc761a","Type":"ContainerStarted","Data":"a2aad1cb2b7a34d3a4853b25fa69afff81381983945e1e5371266de85a47740e"}
Jul 13 22:04:59.223058 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:04:59.223003    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-766bd547bb-vbd5d" podStartSLOduration=86.654338888 podStartE2EDuration="1m30.222987956s" totalImagesPullingTime="3.568649068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:03:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:03:33.974389668 +0000 UTC m=+301.892366134" watchObservedRunningTime="2026-07-13 22:04:59.222987956 +0000 UTC m=+387.140964430"
Jul 13 22:04:59.224396 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:04:59.224371    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"]
Jul 13 22:04:59.227780 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:04:59.227759    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"
Jul 13 22:04:59.231406 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:04:59.231381    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-g5rnv\""
Jul 13 22:04:59.231505 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:04:59.231386    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 22:04:59.234535 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:04:59.234520    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 22:04:59.243909 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:04:59.243886    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"]
Jul 13 22:04:59.326292 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:04:59.326252    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkzdt\" (UniqueName: \"kubernetes.io/projected/61ec21e1-6cbb-4905-a8e3-31f5becb21e1-kube-api-access-hkzdt\") pod \"limitador-operator-controller-manager-859d7b78bd-tq6t8\" (UID: \"61ec21e1-6cbb-4905-a8e3-31f5becb21e1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"
Jul 13 22:04:59.426898 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:04:59.426868    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkzdt\" (UniqueName: \"kubernetes.io/projected/61ec21e1-6cbb-4905-a8e3-31f5becb21e1-kube-api-access-hkzdt\") pod \"limitador-operator-controller-manager-859d7b78bd-tq6t8\" (UID: \"61ec21e1-6cbb-4905-a8e3-31f5becb21e1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"
Jul 13 22:04:59.446453 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:04:59.446422    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkzdt\" (UniqueName: \"kubernetes.io/projected/61ec21e1-6cbb-4905-a8e3-31f5becb21e1-kube-api-access-hkzdt\") pod \"limitador-operator-controller-manager-859d7b78bd-tq6t8\" (UID: \"61ec21e1-6cbb-4905-a8e3-31f5becb21e1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"
Jul 13 22:04:59.538780 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:04:59.538752    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"
Jul 13 22:04:59.694160 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:04:59.694005    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"]
Jul 13 22:04:59.696858 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:04:59.696834    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61ec21e1_6cbb_4905_a8e3_31f5becb21e1.slice/crio-2a7136c842c5aac2e18efc786d1630f81f9cbd27171c57dbfbdd9de13a9c8be2 WatchSource:0}: Error finding container 2a7136c842c5aac2e18efc786d1630f81f9cbd27171c57dbfbdd9de13a9c8be2: Status 404 returned error can't find the container with id 2a7136c842c5aac2e18efc786d1630f81f9cbd27171c57dbfbdd9de13a9c8be2
Jul 13 22:04:59.698839 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:04:59.698822    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 22:05:00.278849 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:00.278810    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8" event={"ID":"61ec21e1-6cbb-4905-a8e3-31f5becb21e1","Type":"ContainerStarted","Data":"2a7136c842c5aac2e18efc786d1630f81f9cbd27171c57dbfbdd9de13a9c8be2"}
Jul 13 22:05:02.289859 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:02.289771    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8" event={"ID":"61ec21e1-6cbb-4905-a8e3-31f5becb21e1","Type":"ContainerStarted","Data":"ce2c749ee2c8938ee1e17c9ab1d5d71966e44988594ce10751ced113ffda2eed"}
Jul 13 22:05:02.290305 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:02.289890    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"
Jul 13 22:05:13.295343 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:13.295314    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"
Jul 13 22:05:13.322733 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:13.322680    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8" podStartSLOduration=11.995795672 podStartE2EDuration="14.322664999s" totalImagesPullingTime="2.326869327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:05:02.312647766 +0000 UTC m=+390.230624229" watchObservedRunningTime="2026-07-13 22:05:13.322664999 +0000 UTC m=+401.240641464"
Jul 13 22:05:14.193416 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.193382    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"]
Jul 13 22:05:14.196532 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.196518    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"
Jul 13 22:05:14.201934 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.201905    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6kkbb\""
Jul 13 22:05:14.202046 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.201989    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 13 22:05:14.202046 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.202004    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 13 22:05:14.216688 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.216667    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"]
Jul 13 22:05:14.240011 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.239988    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c377887a-d696-4434-9f6c-3d3e621e5a26-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rr499\" (UID: \"c377887a-d696-4434-9f6c-3d3e621e5a26\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"
Jul 13 22:05:14.240108 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.240013    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp9s5\" (UniqueName: \"kubernetes.io/projected/c377887a-d696-4434-9f6c-3d3e621e5a26-kube-api-access-hp9s5\") pod \"kuadrant-console-plugin-66b5774985-rr499\" (UID: \"c377887a-d696-4434-9f6c-3d3e621e5a26\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"
Jul 13 22:05:14.240108 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.240092    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c377887a-d696-4434-9f6c-3d3e621e5a26-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rr499\" (UID: \"c377887a-d696-4434-9f6c-3d3e621e5a26\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"
Jul 13 22:05:14.340870 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.340844    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c377887a-d696-4434-9f6c-3d3e621e5a26-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rr499\" (UID: \"c377887a-d696-4434-9f6c-3d3e621e5a26\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"
Jul 13 22:05:14.340870 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.340873    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp9s5\" (UniqueName: \"kubernetes.io/projected/c377887a-d696-4434-9f6c-3d3e621e5a26-kube-api-access-hp9s5\") pod \"kuadrant-console-plugin-66b5774985-rr499\" (UID: \"c377887a-d696-4434-9f6c-3d3e621e5a26\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"
Jul 13 22:05:14.341216 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.340924    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c377887a-d696-4434-9f6c-3d3e621e5a26-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rr499\" (UID: \"c377887a-d696-4434-9f6c-3d3e621e5a26\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"
Jul 13 22:05:14.341459 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.341440    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c377887a-d696-4434-9f6c-3d3e621e5a26-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rr499\" (UID: \"c377887a-d696-4434-9f6c-3d3e621e5a26\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"
Jul 13 22:05:14.343283 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.343264    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c377887a-d696-4434-9f6c-3d3e621e5a26-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rr499\" (UID: \"c377887a-d696-4434-9f6c-3d3e621e5a26\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"
Jul 13 22:05:14.351666 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.351642    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp9s5\" (UniqueName: \"kubernetes.io/projected/c377887a-d696-4434-9f6c-3d3e621e5a26-kube-api-access-hp9s5\") pod \"kuadrant-console-plugin-66b5774985-rr499\" (UID: \"c377887a-d696-4434-9f6c-3d3e621e5a26\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"
Jul 13 22:05:14.504998 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.504973    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"
Jul 13 22:05:14.649249 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:14.649216    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rr499"]
Jul 13 22:05:14.652722 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:05:14.652695    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc377887a_d696_4434_9f6c_3d3e621e5a26.slice/crio-088ab3c41f033ffa75618e591e3de4b8d7b8ec70b182908817c0ca77bd88d3f9 WatchSource:0}: Error finding container 088ab3c41f033ffa75618e591e3de4b8d7b8ec70b182908817c0ca77bd88d3f9: Status 404 returned error can't find the container with id 088ab3c41f033ffa75618e591e3de4b8d7b8ec70b182908817c0ca77bd88d3f9
Jul 13 22:05:15.341195 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:15.341160    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499" event={"ID":"c377887a-d696-4434-9f6c-3d3e621e5a26","Type":"ContainerStarted","Data":"088ab3c41f033ffa75618e591e3de4b8d7b8ec70b182908817c0ca77bd88d3f9"}
Jul 13 22:05:24.953040 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:24.953002    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"]
Jul 13 22:05:24.953494 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:24.953289    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8" podUID="61ec21e1-6cbb-4905-a8e3-31f5becb21e1" containerName="manager" containerID="cri-o://ce2c749ee2c8938ee1e17c9ab1d5d71966e44988594ce10751ced113ffda2eed" gracePeriod=2
Jul 13 22:05:24.960688 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:24.960664    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"]
Jul 13 22:05:25.090029 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:25.089995    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztb7q"]
Jul 13 22:05:25.090441 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:25.090425    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61ec21e1-6cbb-4905-a8e3-31f5becb21e1" containerName="manager"
Jul 13 22:05:25.090441 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:25.090443    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="61ec21e1-6cbb-4905-a8e3-31f5becb21e1" containerName="manager"
Jul 13 22:05:25.090606 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:25.090506    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="61ec21e1-6cbb-4905-a8e3-31f5becb21e1" containerName="manager"
Jul 13 22:05:25.094402 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:25.094380    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztb7q"
Jul 13 22:05:25.124813 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:05:25.124779    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tq6t8\" is forbidden: User \"system:node:ip-10-0-143-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-101.ec2.internal' and this object" podUID="61ec21e1-6cbb-4905-a8e3-31f5becb21e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"
Jul 13 22:05:25.149041 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:25.149015    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztb7q"]
Jul 13 22:05:25.240162 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:25.240117    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhsct\" (UniqueName: \"kubernetes.io/projected/52f2613e-f7a2-4c13-b472-d22231443c1c-kube-api-access-xhsct\") pod \"limitador-operator-controller-manager-859d7b78bd-ztb7q\" (UID: \"52f2613e-f7a2-4c13-b472-d22231443c1c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztb7q"
Jul 13 22:05:25.340995 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:25.340962    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhsct\" (UniqueName: \"kubernetes.io/projected/52f2613e-f7a2-4c13-b472-d22231443c1c-kube-api-access-xhsct\") pod \"limitador-operator-controller-manager-859d7b78bd-ztb7q\" (UID: \"52f2613e-f7a2-4c13-b472-d22231443c1c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztb7q"
Jul 13 22:05:25.350322 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:25.350298    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhsct\" (UniqueName: \"kubernetes.io/projected/52f2613e-f7a2-4c13-b472-d22231443c1c-kube-api-access-xhsct\") pod \"limitador-operator-controller-manager-859d7b78bd-ztb7q\" (UID: \"52f2613e-f7a2-4c13-b472-d22231443c1c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztb7q"
Jul 13 22:05:25.406428 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:25.406401    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztb7q"
Jul 13 22:05:38.257110 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.257086    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"
Jul 13 22:05:38.259331 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:05:38.259309    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tq6t8\" is forbidden: User \"system:node:ip-10-0-143-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-101.ec2.internal' and this object" podUID="61ec21e1-6cbb-4905-a8e3-31f5becb21e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"
Jul 13 22:05:38.269322 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.267943    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztb7q"]
Jul 13 22:05:38.272466 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:05:38.272439    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f2613e_f7a2_4c13_b472_d22231443c1c.slice/crio-a60b5fd8f49de5a5b95d698c440bdf22278baefa2c83b884452c5754f1e848ee WatchSource:0}: Error finding container a60b5fd8f49de5a5b95d698c440bdf22278baefa2c83b884452c5754f1e848ee: Status 404 returned error can't find the container with id a60b5fd8f49de5a5b95d698c440bdf22278baefa2c83b884452c5754f1e848ee
Jul 13 22:05:38.352838 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.352804    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkzdt\" (UniqueName: \"kubernetes.io/projected/61ec21e1-6cbb-4905-a8e3-31f5becb21e1-kube-api-access-hkzdt\") pod \"61ec21e1-6cbb-4905-a8e3-31f5becb21e1\" (UID: \"61ec21e1-6cbb-4905-a8e3-31f5becb21e1\") "
Jul 13 22:05:38.354627 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.354603    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61ec21e1-6cbb-4905-a8e3-31f5becb21e1-kube-api-access-hkzdt" (OuterVolumeSpecName: "kube-api-access-hkzdt") pod "61ec21e1-6cbb-4905-a8e3-31f5becb21e1" (UID: "61ec21e1-6cbb-4905-a8e3-31f5becb21e1"). InnerVolumeSpecName "kube-api-access-hkzdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:05:38.453358 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.453294    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztb7q" event={"ID":"52f2613e-f7a2-4c13-b472-d22231443c1c","Type":"ContainerStarted","Data":"01d99e8dca19200170a3063b8b4a779f12579c477f4cf2b559de0fcc4a436f0c"}
Jul 13 22:05:38.453358 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.453342    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztb7q" event={"ID":"52f2613e-f7a2-4c13-b472-d22231443c1c","Type":"ContainerStarted","Data":"a60b5fd8f49de5a5b95d698c440bdf22278baefa2c83b884452c5754f1e848ee"}
Jul 13 22:05:38.453358 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.453342    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkzdt\" (UniqueName: \"kubernetes.io/projected/61ec21e1-6cbb-4905-a8e3-31f5becb21e1-kube-api-access-hkzdt\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:05:38.453573 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.453405    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztb7q"
Jul 13 22:05:38.454672 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.454644    2558 generic.go:356] "Generic (PLEG): container finished" podID="61ec21e1-6cbb-4905-a8e3-31f5becb21e1" containerID="ce2c749ee2c8938ee1e17c9ab1d5d71966e44988594ce10751ced113ffda2eed" exitCode=0
Jul 13 22:05:38.454774 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.454681    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"
Jul 13 22:05:38.454774 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.454737    2558 scope.go:117] "RemoveContainer" containerID="ce2c749ee2c8938ee1e17c9ab1d5d71966e44988594ce10751ced113ffda2eed"
Jul 13 22:05:38.455667 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:05:38.455619    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tq6t8\" is forbidden: User \"system:node:ip-10-0-143-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-101.ec2.internal' and this object" podUID="61ec21e1-6cbb-4905-a8e3-31f5becb21e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"
Jul 13 22:05:38.456444 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.456415    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499" event={"ID":"c377887a-d696-4434-9f6c-3d3e621e5a26","Type":"ContainerStarted","Data":"a800d6f191ea4918833c43f19f63e4e56579ac6d02edb2e779da56fc27b6ee28"}
Jul 13 22:05:38.464952 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.464934    2558 scope.go:117] "RemoveContainer" containerID="ce2c749ee2c8938ee1e17c9ab1d5d71966e44988594ce10751ced113ffda2eed"
Jul 13 22:05:38.465187 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:05:38.465169    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce2c749ee2c8938ee1e17c9ab1d5d71966e44988594ce10751ced113ffda2eed\": container with ID starting with ce2c749ee2c8938ee1e17c9ab1d5d71966e44988594ce10751ced113ffda2eed not found: ID does not exist" containerID="ce2c749ee2c8938ee1e17c9ab1d5d71966e44988594ce10751ced113ffda2eed"
Jul 13 22:05:38.465231 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.465195    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2c749ee2c8938ee1e17c9ab1d5d71966e44988594ce10751ced113ffda2eed"} err="failed to get container status \"ce2c749ee2c8938ee1e17c9ab1d5d71966e44988594ce10751ced113ffda2eed\": rpc error: code = NotFound desc = could not find container \"ce2c749ee2c8938ee1e17c9ab1d5d71966e44988594ce10751ced113ffda2eed\": container with ID starting with ce2c749ee2c8938ee1e17c9ab1d5d71966e44988594ce10751ced113ffda2eed not found: ID does not exist"
Jul 13 22:05:38.476083 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.476049    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztb7q" podStartSLOduration=13.476038528 podStartE2EDuration="13.476038528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:05:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:05:38.474958205 +0000 UTC m=+426.392934671" watchObservedRunningTime="2026-07-13 22:05:38.476038528 +0000 UTC m=+426.394014992"
Jul 13 22:05:38.477140 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:05:38.477111    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tq6t8\" is forbidden: User \"system:node:ip-10-0-143-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-101.ec2.internal' and this object" podUID="61ec21e1-6cbb-4905-a8e3-31f5becb21e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tq6t8"
Jul 13 22:05:38.493920 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.493817    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rr499" podStartSLOduration=0.938010848 podStartE2EDuration="24.49380576s" totalImagesPullingTime="23.555794912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:05:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:05:38.493674662 +0000 UTC m=+426.411651127" watchObservedRunningTime="2026-07-13 22:05:38.49380576 +0000 UTC m=+426.411782224"
Jul 13 22:05:38.665166 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:38.665137    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61ec21e1-6cbb-4905-a8e3-31f5becb21e1" path="/var/lib/kubelet/pods/61ec21e1-6cbb-4905-a8e3-31f5becb21e1/volumes"
Jul 13 22:05:49.463485 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:49.463456    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztb7q"
Jul 13 22:05:55.668509 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:55.668439    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:05:55.670743 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:55.670726    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"
Jul 13 22:05:55.673472 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:55.673453    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 13 22:05:55.684603 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:55.684583    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:05:55.717460 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:55.717438    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:05:55.777623 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:55.777598    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nlc5\" (UniqueName: \"kubernetes.io/projected/ebf4ce38-1577-4758-aa22-2331d2a89221-kube-api-access-6nlc5\") pod \"limitador-limitador-54d6d988d5-7r2wx\" (UID: \"ebf4ce38-1577-4758-aa22-2331d2a89221\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"
Jul 13 22:05:55.777719 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:55.777646    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ebf4ce38-1577-4758-aa22-2331d2a89221-config-file\") pod \"limitador-limitador-54d6d988d5-7r2wx\" (UID: \"ebf4ce38-1577-4758-aa22-2331d2a89221\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"
Jul 13 22:05:55.879074 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:55.879042    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nlc5\" (UniqueName: \"kubernetes.io/projected/ebf4ce38-1577-4758-aa22-2331d2a89221-kube-api-access-6nlc5\") pod \"limitador-limitador-54d6d988d5-7r2wx\" (UID: \"ebf4ce38-1577-4758-aa22-2331d2a89221\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"
Jul 13 22:05:55.879213 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:55.879085    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ebf4ce38-1577-4758-aa22-2331d2a89221-config-file\") pod \"limitador-limitador-54d6d988d5-7r2wx\" (UID: \"ebf4ce38-1577-4758-aa22-2331d2a89221\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"
Jul 13 22:05:55.879677 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:55.879661    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ebf4ce38-1577-4758-aa22-2331d2a89221-config-file\") pod \"limitador-limitador-54d6d988d5-7r2wx\" (UID: \"ebf4ce38-1577-4758-aa22-2331d2a89221\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"
Jul 13 22:05:55.886550 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:55.886519    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nlc5\" (UniqueName: \"kubernetes.io/projected/ebf4ce38-1577-4758-aa22-2331d2a89221-kube-api-access-6nlc5\") pod \"limitador-limitador-54d6d988d5-7r2wx\" (UID: \"ebf4ce38-1577-4758-aa22-2331d2a89221\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"
Jul 13 22:05:55.980129 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:55.980111    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"
Jul 13 22:05:56.108789 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:56.108748    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:05:56.111889 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:05:56.111865    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf4ce38_1577_4758_aa22_2331d2a89221.slice/crio-7a711cf97c8edef378728c2369e903ee4958e2d976a1ec23279d5585c674d2c6 WatchSource:0}: Error finding container 7a711cf97c8edef378728c2369e903ee4958e2d976a1ec23279d5585c674d2c6: Status 404 returned error can't find the container with id 7a711cf97c8edef378728c2369e903ee4958e2d976a1ec23279d5585c674d2c6
Jul 13 22:05:56.208535 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:56.208493    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-nhsvx"]
Jul 13 22:05:56.211734 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:56.211714    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nhsvx"
Jul 13 22:05:56.214329 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:56.214311    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-89wkk\""
Jul 13 22:05:56.218730 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:56.218709    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nhsvx"]
Jul 13 22:05:56.282179 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:56.282127    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp9gd\" (UniqueName: \"kubernetes.io/projected/f8d1d9da-ae00-4900-b6db-9d14e11baa20-kube-api-access-mp9gd\") pod \"authorino-75d556bd77-nhsvx\" (UID: \"f8d1d9da-ae00-4900-b6db-9d14e11baa20\") " pod="kuadrant-system/authorino-75d556bd77-nhsvx"
Jul 13 22:05:56.382814 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:56.382790    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp9gd\" (UniqueName: \"kubernetes.io/projected/f8d1d9da-ae00-4900-b6db-9d14e11baa20-kube-api-access-mp9gd\") pod \"authorino-75d556bd77-nhsvx\" (UID: \"f8d1d9da-ae00-4900-b6db-9d14e11baa20\") " pod="kuadrant-system/authorino-75d556bd77-nhsvx"
Jul 13 22:05:56.390781 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:56.390760    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp9gd\" (UniqueName: \"kubernetes.io/projected/f8d1d9da-ae00-4900-b6db-9d14e11baa20-kube-api-access-mp9gd\") pod \"authorino-75d556bd77-nhsvx\" (UID: \"f8d1d9da-ae00-4900-b6db-9d14e11baa20\") " pod="kuadrant-system/authorino-75d556bd77-nhsvx"
Jul 13 22:05:56.521105 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:56.521083    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nhsvx"
Jul 13 22:05:56.531372 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:56.531337    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-7r2wx" event={"ID":"ebf4ce38-1577-4758-aa22-2331d2a89221","Type":"ContainerStarted","Data":"7a711cf97c8edef378728c2369e903ee4958e2d976a1ec23279d5585c674d2c6"}
Jul 13 22:05:56.642427 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:56.642402    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nhsvx"]
Jul 13 22:05:56.644747 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:05:56.644720    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8d1d9da_ae00_4900_b6db_9d14e11baa20.slice/crio-95bc4bee83e54ce6bc7f76f1fdba52365ecfa18308cfd2fc0ee06171cba4e565 WatchSource:0}: Error finding container 95bc4bee83e54ce6bc7f76f1fdba52365ecfa18308cfd2fc0ee06171cba4e565: Status 404 returned error can't find the container with id 95bc4bee83e54ce6bc7f76f1fdba52365ecfa18308cfd2fc0ee06171cba4e565
Jul 13 22:05:57.545916 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:05:57.545847    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nhsvx" event={"ID":"f8d1d9da-ae00-4900-b6db-9d14e11baa20","Type":"ContainerStarted","Data":"95bc4bee83e54ce6bc7f76f1fdba52365ecfa18308cfd2fc0ee06171cba4e565"}
Jul 13 22:06:02.568943 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:02.568900    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-7r2wx" event={"ID":"ebf4ce38-1577-4758-aa22-2331d2a89221","Type":"ContainerStarted","Data":"a6b6cb05ae9e108a5baf2e537704fe0a8d20c48ddb36cda1593b7ef60845b79a"}
Jul 13 22:06:02.569369 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:02.568964    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"
Jul 13 22:06:02.570408 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:02.570377    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nhsvx" event={"ID":"f8d1d9da-ae00-4900-b6db-9d14e11baa20","Type":"ContainerStarted","Data":"570017d982930172e6dd608f8fa8d73b12fccdbc2cbb427438e15b5e01d5f005"}
Jul 13 22:06:02.585549 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:02.585503    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-7r2wx" podStartSLOduration=2.002289488 podStartE2EDuration="7.585489991s" totalImagesPullingTime="5.583200503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:06:02.58391305 +0000 UTC m=+450.501889510" watchObservedRunningTime="2026-07-13 22:06:02.585489991 +0000 UTC m=+450.503466458"
Jul 13 22:06:02.597871 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:02.597828    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-nhsvx" podStartSLOduration=1.55461765 podStartE2EDuration="6.597814651s" totalImagesPullingTime="5.043197001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:05:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:06:02.596870152 +0000 UTC m=+450.514846636" watchObservedRunningTime="2026-07-13 22:06:02.597814651 +0000 UTC m=+450.515791115"
Jul 13 22:06:13.576397 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:13.576364    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"
Jul 13 22:06:31.500280 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:31.500249    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nhsvx"]
Jul 13 22:06:31.500695 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:31.500423    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-nhsvx" podUID="f8d1d9da-ae00-4900-b6db-9d14e11baa20" containerName="authorino" containerID="cri-o://570017d982930172e6dd608f8fa8d73b12fccdbc2cbb427438e15b5e01d5f005" gracePeriod=30
Jul 13 22:06:31.695415 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:31.695380    2558 generic.go:356] "Generic (PLEG): container finished" podID="f8d1d9da-ae00-4900-b6db-9d14e11baa20" containerID="570017d982930172e6dd608f8fa8d73b12fccdbc2cbb427438e15b5e01d5f005" exitCode=0
Jul 13 22:06:31.695532 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:31.695455    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nhsvx" event={"ID":"f8d1d9da-ae00-4900-b6db-9d14e11baa20","Type":"ContainerDied","Data":"570017d982930172e6dd608f8fa8d73b12fccdbc2cbb427438e15b5e01d5f005"}
Jul 13 22:06:31.739098 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:31.739076    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nhsvx"
Jul 13 22:06:31.855829 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:31.855775    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp9gd\" (UniqueName: \"kubernetes.io/projected/f8d1d9da-ae00-4900-b6db-9d14e11baa20-kube-api-access-mp9gd\") pod \"f8d1d9da-ae00-4900-b6db-9d14e11baa20\" (UID: \"f8d1d9da-ae00-4900-b6db-9d14e11baa20\") "
Jul 13 22:06:31.857686 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:31.857658    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8d1d9da-ae00-4900-b6db-9d14e11baa20-kube-api-access-mp9gd" (OuterVolumeSpecName: "kube-api-access-mp9gd") pod "f8d1d9da-ae00-4900-b6db-9d14e11baa20" (UID: "f8d1d9da-ae00-4900-b6db-9d14e11baa20"). InnerVolumeSpecName "kube-api-access-mp9gd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:06:31.956532 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:31.956510    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp9gd\" (UniqueName: \"kubernetes.io/projected/f8d1d9da-ae00-4900-b6db-9d14e11baa20-kube-api-access-mp9gd\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:06:32.700319 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:32.700298    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nhsvx"
Jul 13 22:06:32.700737 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:32.700332    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nhsvx" event={"ID":"f8d1d9da-ae00-4900-b6db-9d14e11baa20","Type":"ContainerDied","Data":"95bc4bee83e54ce6bc7f76f1fdba52365ecfa18308cfd2fc0ee06171cba4e565"}
Jul 13 22:06:32.700737 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:32.700380    2558 scope.go:117] "RemoveContainer" containerID="570017d982930172e6dd608f8fa8d73b12fccdbc2cbb427438e15b5e01d5f005"
Jul 13 22:06:32.715951 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:32.715929    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nhsvx"]
Jul 13 22:06:32.721003 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:32.720980    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nhsvx"]
Jul 13 22:06:34.665237 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:06:34.665206    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8d1d9da-ae00-4900-b6db-9d14e11baa20" path="/var/lib/kubelet/pods/f8d1d9da-ae00-4900-b6db-9d14e11baa20/volumes"
Jul 13 22:07:15.223335 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:07:15.223304    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:07:33.464040 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:07:33.464008    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:07:48.353595 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:07:48.353561    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:08:20.024130 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:20.024093    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:08:25.852692 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:25.852657    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:08:29.645098 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.645064    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"]
Jul 13 22:08:29.645443 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.645383    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8d1d9da-ae00-4900-b6db-9d14e11baa20" containerName="authorino"
Jul 13 22:08:29.645443 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.645394    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8d1d9da-ae00-4900-b6db-9d14e11baa20" containerName="authorino"
Jul 13 22:08:29.645510 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.645446    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8d1d9da-ae00-4900-b6db-9d14e11baa20" containerName="authorino"
Jul 13 22:08:29.648373 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.648357    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.650891 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.650871    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 13 22:08:29.651828 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.651812    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 13 22:08:29.651946 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.651910    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2h8t7\""
Jul 13 22:08:29.651946 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.651929    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 13 22:08:29.660384 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.660363    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"]
Jul 13 22:08:29.727167 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.727141    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.727299 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.727193    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.727299 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.727232    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.727299 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.727256    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.727299 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.727294    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfs64\" (UniqueName: \"kubernetes.io/projected/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-kube-api-access-qfs64\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.727462 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.727314    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.827724 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.827693    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.827724 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.827728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.827914 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.827752    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.827914 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.827778    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfs64\" (UniqueName: \"kubernetes.io/projected/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-kube-api-access-qfs64\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.827914 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.827796    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.827914 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.827818    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.828157 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.828133    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.828225 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.828149    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.828278 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.828235    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.830117 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.830102    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.830221 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.830204    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.836454 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.836433    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfs64\" (UniqueName: \"kubernetes.io/projected/b0da60a9-14dc-4fc9-8c87-7da5b0afdeab-kube-api-access-qfs64\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gtf7j\" (UID: \"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:29.957915 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:29.957852    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:30.091625 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:30.091601    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"]
Jul 13 22:08:30.093905 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:08:30.093870    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0da60a9_14dc_4fc9_8c87_7da5b0afdeab.slice/crio-5368bbfefe0deaeb6ca569676ab3dd6639b8481687c913cce105dadae5e6cfb3 WatchSource:0}: Error finding container 5368bbfefe0deaeb6ca569676ab3dd6639b8481687c913cce105dadae5e6cfb3: Status 404 returned error can't find the container with id 5368bbfefe0deaeb6ca569676ab3dd6639b8481687c913cce105dadae5e6cfb3
Jul 13 22:08:30.192430 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:30.192396    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j" event={"ID":"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab","Type":"ContainerStarted","Data":"5368bbfefe0deaeb6ca569676ab3dd6639b8481687c913cce105dadae5e6cfb3"}
Jul 13 22:08:31.198106 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:31.198075    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j" event={"ID":"b0da60a9-14dc-4fc9-8c87-7da5b0afdeab","Type":"ContainerStarted","Data":"3b1dc16f64a446dba497716b6905c9629fbe03bb208ccbf3cd9af785d84a13b0"}
Jul 13 22:08:31.217453 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:31.217405    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j" podStartSLOduration=1.219837079 podStartE2EDuration="2.217389652s" totalImagesPullingTime="997.552573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:08:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:08:31.215414082 +0000 UTC m=+599.133390546" watchObservedRunningTime="2026-07-13 22:08:31.217389652 +0000 UTC m=+599.135366117"
Jul 13 22:08:32.137122 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:32.137074    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:08:32.566342 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:32.566313    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:08:32.568461 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:32.568439    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:08:39.958305 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:39.958270    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:39.960809 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:39.958341    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:39.971170 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:39.971148    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:40.251282 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:40.251253    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gtf7j"
Jul 13 22:08:45.158659 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.158628    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"]
Jul 13 22:08:45.161445 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.161430    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.164855 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.164834    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 13 22:08:45.181598 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.181578    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"]
Jul 13 22:08:45.242730 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.242706    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.242845 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.242739    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.242845 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.242762    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.242845 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.242814    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.242990 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.242846    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.242990 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.242870    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbzf6\" (UniqueName: \"kubernetes.io/projected/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-kube-api-access-rbzf6\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.343633 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.343608    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.343756 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.343643    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.343756 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.343663    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.343756 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.343689    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.343756 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.343718    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.343756 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.343748    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbzf6\" (UniqueName: \"kubernetes.io/projected/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-kube-api-access-rbzf6\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.344104 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.344083    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.344172 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.344117    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.344209 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.344182    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.345804 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.345781    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.346074 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.346058    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.368360 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.368334    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbzf6\" (UniqueName: \"kubernetes.io/projected/1b0cab3b-63be-45e0-8a02-8cb0afbb26fd-kube-api-access-rbzf6\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2\" (UID: \"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.469834 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.469771    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:45.619862 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:45.619831    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"]
Jul 13 22:08:45.622575 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:08:45.622526    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b0cab3b_63be_45e0_8a02_8cb0afbb26fd.slice/crio-94a6c9f266eee22c021a3a82267134718fbd669c44ef971b907e499230752c2b WatchSource:0}: Error finding container 94a6c9f266eee22c021a3a82267134718fbd669c44ef971b907e499230752c2b: Status 404 returned error can't find the container with id 94a6c9f266eee22c021a3a82267134718fbd669c44ef971b907e499230752c2b
Jul 13 22:08:46.266938 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:46.266908    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2" event={"ID":"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd","Type":"ContainerStarted","Data":"8f8b2ea43a8f7093e27dbb1b02876bce4ec0f4ddcf987257fd8a70dc6ef11ed7"}
Jul 13 22:08:46.267268 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:46.266949    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2" event={"ID":"1b0cab3b-63be-45e0-8a02-8cb0afbb26fd","Type":"ContainerStarted","Data":"94a6c9f266eee22c021a3a82267134718fbd669c44ef971b907e499230752c2b"}
Jul 13 22:08:46.291458 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:46.291415    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2" podStartSLOduration=1.036121122 podStartE2EDuration="1.291401116s" totalImagesPullingTime="255.279994ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:08:46.289473343 +0000 UTC m=+614.207449808" watchObservedRunningTime="2026-07-13 22:08:46.291401116 +0000 UTC m=+614.209377579"
Jul 13 22:08:48.153367 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:48.153335    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:08:55.470416 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:55.470336    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:55.470416 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:55.470401    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:55.482773 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:55.482754    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:08:56.320143 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:08:56.320119    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2"
Jul 13 22:09:39.975487 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:39.975455    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65d7d75cf8-5fbt8"]
Jul 13 22:09:39.978761 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:39.978742    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d7d75cf8-5fbt8"
Jul 13 22:09:39.983118 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:39.983098    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-89wkk\""
Jul 13 22:09:39.983204 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:39.983143    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 13 22:09:39.988999 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:39.988981    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65d7d75cf8-5fbt8"]
Jul 13 22:09:40.076414 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:40.076378    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0956ddbd-8f58-41ee-b1df-24d976b0d677-tls-cert\") pod \"authorino-65d7d75cf8-5fbt8\" (UID: \"0956ddbd-8f58-41ee-b1df-24d976b0d677\") " pod="kuadrant-system/authorino-65d7d75cf8-5fbt8"
Jul 13 22:09:40.076510 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:40.076443    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p74sp\" (UniqueName: \"kubernetes.io/projected/0956ddbd-8f58-41ee-b1df-24d976b0d677-kube-api-access-p74sp\") pod \"authorino-65d7d75cf8-5fbt8\" (UID: \"0956ddbd-8f58-41ee-b1df-24d976b0d677\") " pod="kuadrant-system/authorino-65d7d75cf8-5fbt8"
Jul 13 22:09:40.176855 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:40.176834    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p74sp\" (UniqueName: \"kubernetes.io/projected/0956ddbd-8f58-41ee-b1df-24d976b0d677-kube-api-access-p74sp\") pod \"authorino-65d7d75cf8-5fbt8\" (UID: \"0956ddbd-8f58-41ee-b1df-24d976b0d677\") " pod="kuadrant-system/authorino-65d7d75cf8-5fbt8"
Jul 13 22:09:40.176956 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:40.176886    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0956ddbd-8f58-41ee-b1df-24d976b0d677-tls-cert\") pod \"authorino-65d7d75cf8-5fbt8\" (UID: \"0956ddbd-8f58-41ee-b1df-24d976b0d677\") " pod="kuadrant-system/authorino-65d7d75cf8-5fbt8"
Jul 13 22:09:40.179321 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:40.179301    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0956ddbd-8f58-41ee-b1df-24d976b0d677-tls-cert\") pod \"authorino-65d7d75cf8-5fbt8\" (UID: \"0956ddbd-8f58-41ee-b1df-24d976b0d677\") " pod="kuadrant-system/authorino-65d7d75cf8-5fbt8"
Jul 13 22:09:40.184994 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:40.184975    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p74sp\" (UniqueName: \"kubernetes.io/projected/0956ddbd-8f58-41ee-b1df-24d976b0d677-kube-api-access-p74sp\") pod \"authorino-65d7d75cf8-5fbt8\" (UID: \"0956ddbd-8f58-41ee-b1df-24d976b0d677\") " pod="kuadrant-system/authorino-65d7d75cf8-5fbt8"
Jul 13 22:09:40.287930 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:40.287907    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d7d75cf8-5fbt8"
Jul 13 22:09:40.415252 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:40.415228    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65d7d75cf8-5fbt8"]
Jul 13 22:09:40.417461 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:09:40.417433    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0956ddbd_8f58_41ee_b1df_24d976b0d677.slice/crio-d58bb359e08c39c3f013a1b3a3f5066e18a9f55fecfbfa09219038ca967e6be7 WatchSource:0}: Error finding container d58bb359e08c39c3f013a1b3a3f5066e18a9f55fecfbfa09219038ca967e6be7: Status 404 returned error can't find the container with id d58bb359e08c39c3f013a1b3a3f5066e18a9f55fecfbfa09219038ca967e6be7
Jul 13 22:09:40.497171 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:40.497143    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d7d75cf8-5fbt8" event={"ID":"0956ddbd-8f58-41ee-b1df-24d976b0d677","Type":"ContainerStarted","Data":"d58bb359e08c39c3f013a1b3a3f5066e18a9f55fecfbfa09219038ca967e6be7"}
Jul 13 22:09:41.504389 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:41.504346    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d7d75cf8-5fbt8" event={"ID":"0956ddbd-8f58-41ee-b1df-24d976b0d677","Type":"ContainerStarted","Data":"649bba343ea231b1617f2dd622ef7ebbc4f8140b8e89e5e7fb34e66b071e8af3"}
Jul 13 22:09:41.537118 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:09:41.537069    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-65d7d75cf8-5fbt8" podStartSLOduration=2.124768144 podStartE2EDuration="2.537050769s" totalImagesPullingTime="412.282625ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:09:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:09:41.53497812 +0000 UTC m=+669.452954586" watchObservedRunningTime="2026-07-13 22:09:41.537050769 +0000 UTC m=+669.455027233"
Jul 13 22:10:09.669563 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:10:09.669516    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:10:19.842953 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:10:19.842904    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:10:29.047156 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:10:29.047079    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:10:39.048474 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:10:39.048437    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:10:49.031839 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:10:49.031806    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:11:15.433809 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:11:15.433769    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:11:29.437816 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:11:29.437780    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:12:03.479445 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:12:03.479374    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:12:13.041784 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:12:13.041751    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:12:23.632827 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:12:23.632794    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:12:29.731207 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:12:29.731175    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:12:40.332442 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:12:40.332409    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:12:50.444612 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:12:50.444584    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:13:01.150291 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:13:01.150257    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:13:11.634660 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:13:11.634629    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:13:32.594701 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:13:32.594677    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:13:32.602329 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:13:32.602310    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:13:33.343818 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:13:33.343783    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:13:43.637909 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:13:43.637866    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:14:06.873179 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:14:06.873141    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:14:17.942147 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:14:17.942113    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:14:34.160390 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:14:34.160349    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:14:45.610028 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:14:45.609996    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:14:55.613687 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:14:55.613603    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:15:05.558977 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:15:05.558947    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:15:15.030744 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:15:15.030712    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:16:59.946105 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:16:59.946072    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:17:08.834893 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:17:08.834861    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:17:21.434726 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:17:21.434691    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:17:32.728659 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:17:32.728620    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:17:43.131400 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:17:43.131364    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:17:54.442279 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:17:54.437495    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:18:03.951855 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:18:03.951828    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:18:15.341007 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:18:15.340970    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:18:25.648966 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:18:25.648935    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:18:32.622961 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:18:32.622922    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:18:32.638067 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:18:32.638041    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:18:34.739852 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:18:34.739820    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:18:46.146044 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:18:46.146011    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:19:11.822836 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:19:11.822804    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:19:19.540675 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:19:19.540634    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:19:29.034610 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:19:29.034517    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:19:38.639354 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:19:38.639318    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:19:56.135693 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:19:56.135663    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:20:06.542176 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:20:06.542141    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:20:15.537742 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:20:15.537691    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:20:27.173984 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:20:27.173946    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:20:35.738833 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:20:35.738800    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:20:45.534560 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:20:45.534506    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:20:58.529932 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:20:58.529855    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:21:07.932918 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:21:07.932883    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:21:17.235949 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:21:17.235916    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:21:26.739266 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:21:26.739232    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:21:39.435792 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:21:39.435757    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:21:48.938495 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:21:48.938453    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:22:01.773655 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:22:01.773626    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:22:13.537240 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:22:13.537205    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:22:26.141143 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:22:26.141068    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:22:39.144120 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:22:39.144089    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:22:47.438123 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:22:47.438087    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:22:57.244501 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:22:57.244463    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:23:15.242911 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:23:15.242877    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:23:27.143043 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:23:27.143010    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:23:32.655498 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:23:32.655449    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:23:32.666308 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:23:32.666289    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:25:05.041457 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:25:05.041423    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:25:14.437989 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:25:14.437951    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:25:28.841087 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:25:28.841014    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:25:39.256606 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:25:39.256577    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:26:08.036152 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:08.036119    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:26:12.943866 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:12.943837    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"]
Jul 13 22:26:12.947005 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:12.946985    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:12.949549 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:12.949515    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6caf7710\"/\"kube-root-ca.crt\""
Jul 13 22:26:12.950532 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:12.950508    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6caf7710\"/\"openshift-service-ca.crt\""
Jul 13 22:26:12.950532 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:12.950518    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6caf7710\"/\"e2e-lifecycle-model-6caf7710-kserve-self-signed-certs\""
Jul 13 22:26:12.950672 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:12.950656    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6caf7710\"/\"default-dockercfg-jgns5\""
Jul 13 22:26:12.958174 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:12.958154    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"]
Jul 13 22:26:13.050565 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.050522    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-model-cache\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.050699 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.050623    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4glcj\" (UniqueName: \"kubernetes.io/projected/34a0fb35-8890-4f4a-aebf-0e493143d1f7-kube-api-access-4glcj\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.050699 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.050650    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34a0fb35-8890-4f4a-aebf-0e493143d1f7-tls-certs\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.050699 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.050670    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-tmp-dir\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.050839 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.050745    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-dshm\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.050839 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.050782    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-home\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.151246 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.151224    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4glcj\" (UniqueName: \"kubernetes.io/projected/34a0fb35-8890-4f4a-aebf-0e493143d1f7-kube-api-access-4glcj\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.151376 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.151262    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34a0fb35-8890-4f4a-aebf-0e493143d1f7-tls-certs\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.151376 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.151290    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-tmp-dir\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.151376 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.151326    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-dshm\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.151608 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.151374    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-home\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.151608 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.151416    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-model-cache\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.151788 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.151770    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-tmp-dir\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.151860 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.151819    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-model-cache\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.151900 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.151861    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-home\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.153557 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.153524    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-dshm\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.153783 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.153760    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34a0fb35-8890-4f4a-aebf-0e493143d1f7-tls-certs\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.159520 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.159498    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4glcj\" (UniqueName: \"kubernetes.io/projected/34a0fb35-8890-4f4a-aebf-0e493143d1f7-kube-api-access-4glcj\") pod \"e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") " pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.256664 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.256626    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:13.385092 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.385067    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"]
Jul 13 22:26:13.387279 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:26:13.387252    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34a0fb35_8890_4f4a_aebf_0e493143d1f7.slice/crio-a4f20d099a20e9e387c223c079c1691d1b3d2a4428582c8750820e6b0ff7aa62 WatchSource:0}: Error finding container a4f20d099a20e9e387c223c079c1691d1b3d2a4428582c8750820e6b0ff7aa62: Status 404 returned error can't find the container with id a4f20d099a20e9e387c223c079c1691d1b3d2a4428582c8750820e6b0ff7aa62
Jul 13 22:26:13.853739 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.853687    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr" event={"ID":"34a0fb35-8890-4f4a-aebf-0e493143d1f7","Type":"ContainerStarted","Data":"e73bd691ab3575d4e1d8aa4850842502dfd6a5e6b8656721fdb830113424167e"}
Jul 13 22:26:13.853902 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.853746    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr" event={"ID":"34a0fb35-8890-4f4a-aebf-0e493143d1f7","Type":"ContainerStarted","Data":"a4f20d099a20e9e387c223c079c1691d1b3d2a4428582c8750820e6b0ff7aa62"}
Jul 13 22:26:13.876928 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:13.876877    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr" podStartSLOduration=1.8768591159999999 podStartE2EDuration="1.876859116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:26:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:26:13.876642423 +0000 UTC m=+1661.794618888" watchObservedRunningTime="2026-07-13 22:26:13.876859116 +0000 UTC m=+1661.794835582"
Jul 13 22:26:23.256970 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:23.256931    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:23.256970 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:23.256984    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:23.268998 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:23.268980    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:23.906004 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:23.905979    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:24.734316 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:24.734283    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:26:34.446743 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:34.446708    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:26:42.250039 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:42.250008    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"]
Jul 13 22:26:47.248177 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:47.248144    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:26:57.269695 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.269607    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr" podUID="34a0fb35-8890-4f4a-aebf-0e493143d1f7" containerName="main" containerID="cri-o://e73bd691ab3575d4e1d8aa4850842502dfd6a5e6b8656721fdb830113424167e" gracePeriod=45
Jul 13 22:26:57.508558 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.508517    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:57.615315 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.615222    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34a0fb35-8890-4f4a-aebf-0e493143d1f7-tls-certs\") pod \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") "
Jul 13 22:26:57.615315 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.615257    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-tmp-dir\") pod \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") "
Jul 13 22:26:57.615315 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.615277    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-home\") pod \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") "
Jul 13 22:26:57.615586 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.615318    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4glcj\" (UniqueName: \"kubernetes.io/projected/34a0fb35-8890-4f4a-aebf-0e493143d1f7-kube-api-access-4glcj\") pod \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") "
Jul 13 22:26:57.615586 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.615344    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-dshm\") pod \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") "
Jul 13 22:26:57.615586 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.615450    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-model-cache\") pod \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\" (UID: \"34a0fb35-8890-4f4a-aebf-0e493143d1f7\") "
Jul 13 22:26:57.615586 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.615575    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "34a0fb35-8890-4f4a-aebf-0e493143d1f7" (UID: "34a0fb35-8890-4f4a-aebf-0e493143d1f7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:26:57.615789 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.615589    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-home" (OuterVolumeSpecName: "home") pod "34a0fb35-8890-4f4a-aebf-0e493143d1f7" (UID: "34a0fb35-8890-4f4a-aebf-0e493143d1f7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:26:57.615789 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.615737    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-tmp-dir\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:26:57.615789 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.615750    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-home\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:26:57.615789 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.615762    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-model-cache" (OuterVolumeSpecName: "model-cache") pod "34a0fb35-8890-4f4a-aebf-0e493143d1f7" (UID: "34a0fb35-8890-4f4a-aebf-0e493143d1f7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:26:57.617581 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.617530    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34a0fb35-8890-4f4a-aebf-0e493143d1f7-kube-api-access-4glcj" (OuterVolumeSpecName: "kube-api-access-4glcj") pod "34a0fb35-8890-4f4a-aebf-0e493143d1f7" (UID: "34a0fb35-8890-4f4a-aebf-0e493143d1f7"). InnerVolumeSpecName "kube-api-access-4glcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:26:57.617581 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.617574    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34a0fb35-8890-4f4a-aebf-0e493143d1f7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "34a0fb35-8890-4f4a-aebf-0e493143d1f7" (UID: "34a0fb35-8890-4f4a-aebf-0e493143d1f7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:26:57.617764 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.617598    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-dshm" (OuterVolumeSpecName: "dshm") pod "34a0fb35-8890-4f4a-aebf-0e493143d1f7" (UID: "34a0fb35-8890-4f4a-aebf-0e493143d1f7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:26:57.716774 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.716749    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-model-cache\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:26:57.716774 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.716770    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34a0fb35-8890-4f4a-aebf-0e493143d1f7-tls-certs\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:26:57.716774 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.716779    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4glcj\" (UniqueName: \"kubernetes.io/projected/34a0fb35-8890-4f4a-aebf-0e493143d1f7-kube-api-access-4glcj\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:26:57.716960 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:57.716788    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34a0fb35-8890-4f4a-aebf-0e493143d1f7-dshm\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:26:58.045205 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:58.045174    2558 generic.go:356] "Generic (PLEG): container finished" podID="34a0fb35-8890-4f4a-aebf-0e493143d1f7" containerID="e73bd691ab3575d4e1d8aa4850842502dfd6a5e6b8656721fdb830113424167e" exitCode=0
Jul 13 22:26:58.045372 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:58.045224    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"
Jul 13 22:26:58.045372 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:58.045225    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr" event={"ID":"34a0fb35-8890-4f4a-aebf-0e493143d1f7","Type":"ContainerDied","Data":"e73bd691ab3575d4e1d8aa4850842502dfd6a5e6b8656721fdb830113424167e"}
Jul 13 22:26:58.045372 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:58.045283    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr" event={"ID":"34a0fb35-8890-4f4a-aebf-0e493143d1f7","Type":"ContainerDied","Data":"a4f20d099a20e9e387c223c079c1691d1b3d2a4428582c8750820e6b0ff7aa62"}
Jul 13 22:26:58.045372 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:58.045301    2558 scope.go:117] "RemoveContainer" containerID="e73bd691ab3575d4e1d8aa4850842502dfd6a5e6b8656721fdb830113424167e"
Jul 13 22:26:58.054489 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:58.054472    2558 scope.go:117] "RemoveContainer" containerID="e73bd691ab3575d4e1d8aa4850842502dfd6a5e6b8656721fdb830113424167e"
Jul 13 22:26:58.054734 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:26:58.054716    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e73bd691ab3575d4e1d8aa4850842502dfd6a5e6b8656721fdb830113424167e\": container with ID starting with e73bd691ab3575d4e1d8aa4850842502dfd6a5e6b8656721fdb830113424167e not found: ID does not exist" containerID="e73bd691ab3575d4e1d8aa4850842502dfd6a5e6b8656721fdb830113424167e"
Jul 13 22:26:58.054797 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:58.054741    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e73bd691ab3575d4e1d8aa4850842502dfd6a5e6b8656721fdb830113424167e"} err="failed to get container status \"e73bd691ab3575d4e1d8aa4850842502dfd6a5e6b8656721fdb830113424167e\": rpc error: code = NotFound desc = could not find container \"e73bd691ab3575d4e1d8aa4850842502dfd6a5e6b8656721fdb830113424167e\": container with ID starting with e73bd691ab3575d4e1d8aa4850842502dfd6a5e6b8656721fdb830113424167e not found: ID does not exist"
Jul 13 22:26:58.067857 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:58.067836    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"]
Jul 13 22:26:58.072188 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:58.072168    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6caf7710/e2e-lifecycle-model-6caf7710-kserve-7c8fcfc779-nqrwr"]
Jul 13 22:26:58.666302 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:26:58.666273    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34a0fb35-8890-4f4a-aebf-0e493143d1f7" path="/var/lib/kubelet/pods/34a0fb35-8890-4f4a-aebf-0e493143d1f7/volumes"
Jul 13 22:27:22.147777 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:27:22.147743    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:27:31.665488 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:27:31.665454    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:27:44.341333 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:27:44.341273    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:28:32.691470 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:32.691445    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:28:32.695746 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:32.695723    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:28:32.748618 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:32.748586    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:28:47.437455 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:47.437417    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:28:53.078472 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.078438    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"]
Jul 13 22:28:53.078921 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.078786    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34a0fb35-8890-4f4a-aebf-0e493143d1f7" containerName="main"
Jul 13 22:28:53.078921 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.078796    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="34a0fb35-8890-4f4a-aebf-0e493143d1f7" containerName="main"
Jul 13 22:28:53.078921 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.078859    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="34a0fb35-8890-4f4a-aebf-0e493143d1f7" containerName="main"
Jul 13 22:28:53.081917 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.081901    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.084885 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:28:53.084852    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"test-model-fd15db-kserve-self-signed-certs\" is forbidden: User \"system:node:ip-10-0-143-101.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-a-fd15db\": no relationship found between node 'ip-10-0-143-101.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-a-fd15db\"/\"test-model-fd15db-kserve-self-signed-certs\"" type="*v1.Secret"
Jul 13 22:28:53.084885 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:28:53.084857    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-143-101.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-a-fd15db\": no relationship found between node 'ip-10-0-143-101.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-a-fd15db\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 13 22:28:53.098887 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:28:53.098861    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" is forbidden: User \"system:node:ip-10-0-143-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-a-fd15db\": no relationship found between node 'ip-10-0-143-101.ec2.internal' and this object" podUID="f5f7483b-b257-4119-93af-b95f58a7972c" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.133808 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:28:53.133781    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-vbbwf\" is forbidden: User \"system:node:ip-10-0-143-101.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-a-fd15db\": no relationship found between node 'ip-10-0-143-101.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-a-fd15db\"/\"default-dockercfg-vbbwf\"" type="*v1.Secret"
Jul 13 22:28:53.149277 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:28:53.149233    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-143-101.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-a-fd15db\": no relationship found between node 'ip-10-0-143-101.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-a-fd15db\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 13 22:28:53.150939 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.150920    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"]
Jul 13 22:28:53.188489 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.188468    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjsss\" (UniqueName: \"kubernetes.io/projected/f5f7483b-b257-4119-93af-b95f58a7972c-kube-api-access-vjsss\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.188605 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.188501    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-dshm\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.188605 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.188585    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5f7483b-b257-4119-93af-b95f58a7972c-tls-certs\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.188707 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.188622    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-home\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.188707 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.188640    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-tmp-dir\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.188835 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.188717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-model-cache\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.289892 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.289867    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-model-cache\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.289994 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.289901    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjsss\" (UniqueName: \"kubernetes.io/projected/f5f7483b-b257-4119-93af-b95f58a7972c-kube-api-access-vjsss\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.289994 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.289928    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-dshm\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.289994 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.289967    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5f7483b-b257-4119-93af-b95f58a7972c-tls-certs\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.289994 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.289986    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-home\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.290186 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.290000    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-tmp-dir\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.290255 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.290237    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-model-cache\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.290301 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.290261    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-home\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.290407 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.290385    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-tmp-dir\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:53.292153 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:53.292132    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-dshm\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:54.171296 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:54.171269    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-fd15db\"/\"openshift-service-ca.crt\""
Jul 13 22:28:54.290868 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:28:54.290836    2558 secret.go:189] Couldn't get secret ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-self-signed-certs: failed to sync secret cache: timed out waiting for the condition
Jul 13 22:28:54.291027 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:28:54.290918    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5f7483b-b257-4119-93af-b95f58a7972c-tls-certs podName:f5f7483b-b257-4119-93af-b95f58a7972c nodeName:}" failed. No retries permitted until 2026-07-13 22:28:54.790901515 +0000 UTC m=+1822.708877959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f5f7483b-b257-4119-93af-b95f58a7972c-tls-certs") pod "test-model-fd15db-kserve-678c7cc5d5-xrs2f" (UID: "f5f7483b-b257-4119-93af-b95f58a7972c") : failed to sync secret cache: timed out waiting for the condition
Jul 13 22:28:54.300228 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:28:54.300206    2558 projected.go:291] Couldn't get configMap ai-tenant-e2e-inf-a-fd15db/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 13 22:28:54.300348 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:28:54.300236    2558 projected.go:196] Error preparing data for projected volume kube-api-access-vjsss for pod ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f: failed to sync configmap cache: timed out waiting for the condition
Jul 13 22:28:54.300348 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:28:54.300278    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5f7483b-b257-4119-93af-b95f58a7972c-kube-api-access-vjsss podName:f5f7483b-b257-4119-93af-b95f58a7972c nodeName:}" failed. No retries permitted until 2026-07-13 22:28:54.800266309 +0000 UTC m=+1822.718242756 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vjsss" (UniqueName: "kubernetes.io/projected/f5f7483b-b257-4119-93af-b95f58a7972c-kube-api-access-vjsss") pod "test-model-fd15db-kserve-678c7cc5d5-xrs2f" (UID: "f5f7483b-b257-4119-93af-b95f58a7972c") : failed to sync configmap cache: timed out waiting for the condition
Jul 13 22:28:54.330843 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:54.330820    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-fd15db\"/\"kube-root-ca.crt\""
Jul 13 22:28:54.372882 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:54.372863    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-fd15db\"/\"test-model-fd15db-kserve-self-signed-certs\""
Jul 13 22:28:54.443533 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:54.443481    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-fd15db\"/\"default-dockercfg-vbbwf\""
Jul 13 22:28:54.804756 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:54.804719    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5f7483b-b257-4119-93af-b95f58a7972c-tls-certs\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:54.804934 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:54.804834    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjsss\" (UniqueName: \"kubernetes.io/projected/f5f7483b-b257-4119-93af-b95f58a7972c-kube-api-access-vjsss\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:54.807203 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:54.807173    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5f7483b-b257-4119-93af-b95f58a7972c-tls-certs\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:54.807356 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:54.807338    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjsss\" (UniqueName: \"kubernetes.io/projected/f5f7483b-b257-4119-93af-b95f58a7972c-kube-api-access-vjsss\") pod \"test-model-fd15db-kserve-678c7cc5d5-xrs2f\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") " pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:54.891479 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:54.891455    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:28:55.024263 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:55.024227    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"]
Jul 13 22:28:55.027365 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:28:55.027334    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5f7483b_b257_4119_93af_b95f58a7972c.slice/crio-ab637fe750ca8fdd4b15aef0446bbe50dba5e9dc4d03bdb8deb862651c0c3c73 WatchSource:0}: Error finding container ab637fe750ca8fdd4b15aef0446bbe50dba5e9dc4d03bdb8deb862651c0c3c73: Status 404 returned error can't find the container with id ab637fe750ca8fdd4b15aef0446bbe50dba5e9dc4d03bdb8deb862651c0c3c73
Jul 13 22:28:55.555725 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:55.555689    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f" event={"ID":"f5f7483b-b257-4119-93af-b95f58a7972c","Type":"ContainerStarted","Data":"4ee36a1bf90322a60889c198176486586f5b0dfb7216e9ed702f12bbdef62001"}
Jul 13 22:28:55.556089 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:55.555737    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f" event={"ID":"f5f7483b-b257-4119-93af-b95f58a7972c","Type":"ContainerStarted","Data":"ab637fe750ca8fdd4b15aef0446bbe50dba5e9dc4d03bdb8deb862651c0c3c73"}
Jul 13 22:28:55.576860 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:55.576812    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f" podStartSLOduration=2.576797386 podStartE2EDuration="2.576797386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:28:55.576425576 +0000 UTC m=+1823.494402041" watchObservedRunningTime="2026-07-13 22:28:55.576797386 +0000 UTC m=+1823.494773878"
Jul 13 22:28:57.341917 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:28:57.341885    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:29:04.892603 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:04.892566    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:29:04.893037 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:04.892617    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:29:04.904949 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:04.904925    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:29:05.612450 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:05.612423    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:29:09.135916 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:09.135875    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:29:26.848899 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.848867    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"]
Jul 13 22:29:26.852284 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.852264    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.855942 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.855908    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-61b7b8\"/\"test-model-61b7b8-kserve-self-signed-certs\""
Jul 13 22:29:26.856056 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.855944    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-61b7b8\"/\"openshift-service-ca.crt\""
Jul 13 22:29:26.856056 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.855975    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-61b7b8\"/\"kube-root-ca.crt\""
Jul 13 22:29:26.856056 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.856014    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-61b7b8\"/\"default-dockercfg-sncjz\""
Jul 13 22:29:26.861246 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.861228    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"]
Jul 13 22:29:26.879813 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.879793    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b38d6234-7c3d-4fa2-a328-da5902cc74b0-tls-certs\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.879919 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.879826    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-home\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.879919 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.879843    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-model-cache\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.879996 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.879933    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-tmp-dir\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.879996 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.879981    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-dshm\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.880057 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.880040    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqmnb\" (UniqueName: \"kubernetes.io/projected/b38d6234-7c3d-4fa2-a328-da5902cc74b0-kube-api-access-sqmnb\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.981120 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.981095    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqmnb\" (UniqueName: \"kubernetes.io/projected/b38d6234-7c3d-4fa2-a328-da5902cc74b0-kube-api-access-sqmnb\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.981120 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.981127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b38d6234-7c3d-4fa2-a328-da5902cc74b0-tls-certs\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.981329 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.981152    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-home\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.981329 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.981178    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-model-cache\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.981329 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.981243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-tmp-dir\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.981329 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.981273    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-dshm\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.981621 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.981602    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-home\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.981712 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.981644    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-model-cache\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.981712 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.981669    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-tmp-dir\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.983689 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.983670    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b38d6234-7c3d-4fa2-a328-da5902cc74b0-tls-certs\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.984082 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.984063    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-dshm\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:26.990078 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:26.990061    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqmnb\" (UniqueName: \"kubernetes.io/projected/b38d6234-7c3d-4fa2-a328-da5902cc74b0-kube-api-access-sqmnb\") pod \"test-model-61b7b8-kserve-5449df4785-9gzkz\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") " pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:27.162534 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:27.162462    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:27.307301 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:27.307258    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"]
Jul 13 22:29:27.311787 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:29:27.311760    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb38d6234_7c3d_4fa2_a328_da5902cc74b0.slice/crio-8b13ff38ca14ac7535f692a4d63f9b6ae0926e417805901fd27df37c30755f01 WatchSource:0}: Error finding container 8b13ff38ca14ac7535f692a4d63f9b6ae0926e417805901fd27df37c30755f01: Status 404 returned error can't find the container with id 8b13ff38ca14ac7535f692a4d63f9b6ae0926e417805901fd27df37c30755f01
Jul 13 22:29:27.697840 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:27.697806    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz" event={"ID":"b38d6234-7c3d-4fa2-a328-da5902cc74b0","Type":"ContainerStarted","Data":"54f2bf037980b897f9b6c3c053c578a11f3c292bf077757f0e2c6ce8c0c2e3d0"}
Jul 13 22:29:27.698000 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:27.697849    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz" event={"ID":"b38d6234-7c3d-4fa2-a328-da5902cc74b0","Type":"ContainerStarted","Data":"8b13ff38ca14ac7535f692a4d63f9b6ae0926e417805901fd27df37c30755f01"}
Jul 13 22:29:27.725287 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:27.725237    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz" podStartSLOduration=1.7252249499999999 podStartE2EDuration="1.72522495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:29:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:29:27.72472847 +0000 UTC m=+1855.642704933" watchObservedRunningTime="2026-07-13 22:29:27.72522495 +0000 UTC m=+1855.643201414"
Jul 13 22:29:30.648787 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:30.648757    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:29:37.163600 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:37.163569    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:37.163600 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:37.163610    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:37.176153 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:37.176131    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:37.751153 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:37.751120    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:29:47.969306 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:29:47.969272    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:30:05.795894 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:05.795816    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"]
Jul 13 22:30:06.720030 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:06.719993    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"]
Jul 13 22:30:10.736811 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:10.736778    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:30:20.816149 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:20.816093    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f" podUID="f5f7483b-b257-4119-93af-b95f58a7972c" containerName="main" containerID="cri-o://4ee36a1bf90322a60889c198176486586f5b0dfb7216e9ed702f12bbdef62001" gracePeriod=45
Jul 13 22:30:21.068728 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.068673    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:30:21.123170 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.123147    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-model-cache\") pod \"f5f7483b-b257-4119-93af-b95f58a7972c\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") "
Jul 13 22:30:21.123329 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.123181    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjsss\" (UniqueName: \"kubernetes.io/projected/f5f7483b-b257-4119-93af-b95f58a7972c-kube-api-access-vjsss\") pod \"f5f7483b-b257-4119-93af-b95f58a7972c\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") "
Jul 13 22:30:21.123329 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.123220    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5f7483b-b257-4119-93af-b95f58a7972c-tls-certs\") pod \"f5f7483b-b257-4119-93af-b95f58a7972c\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") "
Jul 13 22:30:21.123329 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.123244    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-tmp-dir\") pod \"f5f7483b-b257-4119-93af-b95f58a7972c\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") "
Jul 13 22:30:21.123496 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.123347    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-home\") pod \"f5f7483b-b257-4119-93af-b95f58a7972c\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") "
Jul 13 22:30:21.123496 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.123376    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-dshm\") pod \"f5f7483b-b257-4119-93af-b95f58a7972c\" (UID: \"f5f7483b-b257-4119-93af-b95f58a7972c\") "
Jul 13 22:30:21.123496 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.123431    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-model-cache" (OuterVolumeSpecName: "model-cache") pod "f5f7483b-b257-4119-93af-b95f58a7972c" (UID: "f5f7483b-b257-4119-93af-b95f58a7972c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:30:21.123496 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.123447    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f5f7483b-b257-4119-93af-b95f58a7972c" (UID: "f5f7483b-b257-4119-93af-b95f58a7972c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:30:21.123689 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.123587    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-home" (OuterVolumeSpecName: "home") pod "f5f7483b-b257-4119-93af-b95f58a7972c" (UID: "f5f7483b-b257-4119-93af-b95f58a7972c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:30:21.123744 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.123727    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-model-cache\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:30:21.123783 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.123749    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-tmp-dir\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:30:21.123783 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.123760    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-home\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:30:21.125436 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.125412    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5f7483b-b257-4119-93af-b95f58a7972c-kube-api-access-vjsss" (OuterVolumeSpecName: "kube-api-access-vjsss") pod "f5f7483b-b257-4119-93af-b95f58a7972c" (UID: "f5f7483b-b257-4119-93af-b95f58a7972c"). InnerVolumeSpecName "kube-api-access-vjsss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:30:21.125522 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.125435    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-dshm" (OuterVolumeSpecName: "dshm") pod "f5f7483b-b257-4119-93af-b95f58a7972c" (UID: "f5f7483b-b257-4119-93af-b95f58a7972c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:30:21.125522 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.125442    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5f7483b-b257-4119-93af-b95f58a7972c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f5f7483b-b257-4119-93af-b95f58a7972c" (UID: "f5f7483b-b257-4119-93af-b95f58a7972c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:30:21.225008 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.224984    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjsss\" (UniqueName: \"kubernetes.io/projected/f5f7483b-b257-4119-93af-b95f58a7972c-kube-api-access-vjsss\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:30:21.225008 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.225007    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5f7483b-b257-4119-93af-b95f58a7972c-tls-certs\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:30:21.225171 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.225016    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5f7483b-b257-4119-93af-b95f58a7972c-dshm\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:30:21.742619 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.742575    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz" podUID="b38d6234-7c3d-4fa2-a328-da5902cc74b0" containerName="main" containerID="cri-o://54f2bf037980b897f9b6c3c053c578a11f3c292bf077757f0e2c6ce8c0c2e3d0" gracePeriod=45
Jul 13 22:30:21.932641 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.932598    2558 generic.go:356] "Generic (PLEG): container finished" podID="b38d6234-7c3d-4fa2-a328-da5902cc74b0" containerID="54f2bf037980b897f9b6c3c053c578a11f3c292bf077757f0e2c6ce8c0c2e3d0" exitCode=0
Jul 13 22:30:21.933017 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.932669    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz" event={"ID":"b38d6234-7c3d-4fa2-a328-da5902cc74b0","Type":"ContainerDied","Data":"54f2bf037980b897f9b6c3c053c578a11f3c292bf077757f0e2c6ce8c0c2e3d0"}
Jul 13 22:30:21.934304 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.934285    2558 generic.go:356] "Generic (PLEG): container finished" podID="f5f7483b-b257-4119-93af-b95f58a7972c" containerID="4ee36a1bf90322a60889c198176486586f5b0dfb7216e9ed702f12bbdef62001" exitCode=0
Jul 13 22:30:21.934415 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.934314    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f" event={"ID":"f5f7483b-b257-4119-93af-b95f58a7972c","Type":"ContainerDied","Data":"4ee36a1bf90322a60889c198176486586f5b0dfb7216e9ed702f12bbdef62001"}
Jul 13 22:30:21.934415 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.934337    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f" event={"ID":"f5f7483b-b257-4119-93af-b95f58a7972c","Type":"ContainerDied","Data":"ab637fe750ca8fdd4b15aef0446bbe50dba5e9dc4d03bdb8deb862651c0c3c73"}
Jul 13 22:30:21.934415 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.934353    2558 scope.go:117] "RemoveContainer" containerID="4ee36a1bf90322a60889c198176486586f5b0dfb7216e9ed702f12bbdef62001"
Jul 13 22:30:21.934415 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.934354    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"
Jul 13 22:30:21.944230 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.944209    2558 scope.go:117] "RemoveContainer" containerID="4ee36a1bf90322a60889c198176486586f5b0dfb7216e9ed702f12bbdef62001"
Jul 13 22:30:21.944517 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:30:21.944494    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ee36a1bf90322a60889c198176486586f5b0dfb7216e9ed702f12bbdef62001\": container with ID starting with 4ee36a1bf90322a60889c198176486586f5b0dfb7216e9ed702f12bbdef62001 not found: ID does not exist" containerID="4ee36a1bf90322a60889c198176486586f5b0dfb7216e9ed702f12bbdef62001"
Jul 13 22:30:21.944634 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.944522    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ee36a1bf90322a60889c198176486586f5b0dfb7216e9ed702f12bbdef62001"} err="failed to get container status \"4ee36a1bf90322a60889c198176486586f5b0dfb7216e9ed702f12bbdef62001\": rpc error: code = NotFound desc = could not find container \"4ee36a1bf90322a60889c198176486586f5b0dfb7216e9ed702f12bbdef62001\": container with ID starting with 4ee36a1bf90322a60889c198176486586f5b0dfb7216e9ed702f12bbdef62001 not found: ID does not exist"
Jul 13 22:30:21.956719 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.956694    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"]
Jul 13 22:30:21.962291 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.962270    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-fd15db/test-model-fd15db-kserve-678c7cc5d5-xrs2f"]
Jul 13 22:30:21.991929 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:21.991909    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:30:22.030801 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.030734    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqmnb\" (UniqueName: \"kubernetes.io/projected/b38d6234-7c3d-4fa2-a328-da5902cc74b0-kube-api-access-sqmnb\") pod \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") "
Jul 13 22:30:22.030801 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.030771    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b38d6234-7c3d-4fa2-a328-da5902cc74b0-tls-certs\") pod \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") "
Jul 13 22:30:22.030801 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.030792    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-model-cache\") pod \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") "
Jul 13 22:30:22.031073 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.030819    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-home\") pod \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") "
Jul 13 22:30:22.031073 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.030852    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-dshm\") pod \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") "
Jul 13 22:30:22.031073 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.030942    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-tmp-dir\") pod \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\" (UID: \"b38d6234-7c3d-4fa2-a328-da5902cc74b0\") "
Jul 13 22:30:22.031217 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.031078    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-model-cache" (OuterVolumeSpecName: "model-cache") pod "b38d6234-7c3d-4fa2-a328-da5902cc74b0" (UID: "b38d6234-7c3d-4fa2-a328-da5902cc74b0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:30:22.031217 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.031128    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-home" (OuterVolumeSpecName: "home") pod "b38d6234-7c3d-4fa2-a328-da5902cc74b0" (UID: "b38d6234-7c3d-4fa2-a328-da5902cc74b0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:30:22.031309 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.031265    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-model-cache\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:30:22.031309 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.031284    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-home\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:30:22.031309 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.031280    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b38d6234-7c3d-4fa2-a328-da5902cc74b0" (UID: "b38d6234-7c3d-4fa2-a328-da5902cc74b0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:30:22.033061 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.033041    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-dshm" (OuterVolumeSpecName: "dshm") pod "b38d6234-7c3d-4fa2-a328-da5902cc74b0" (UID: "b38d6234-7c3d-4fa2-a328-da5902cc74b0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:30:22.033317 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.033300    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b38d6234-7c3d-4fa2-a328-da5902cc74b0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b38d6234-7c3d-4fa2-a328-da5902cc74b0" (UID: "b38d6234-7c3d-4fa2-a328-da5902cc74b0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:30:22.033364 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.033334    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b38d6234-7c3d-4fa2-a328-da5902cc74b0-kube-api-access-sqmnb" (OuterVolumeSpecName: "kube-api-access-sqmnb") pod "b38d6234-7c3d-4fa2-a328-da5902cc74b0" (UID: "b38d6234-7c3d-4fa2-a328-da5902cc74b0"). InnerVolumeSpecName "kube-api-access-sqmnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:30:22.131953 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.131927    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-tmp-dir\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:30:22.131953 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.131951    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqmnb\" (UniqueName: \"kubernetes.io/projected/b38d6234-7c3d-4fa2-a328-da5902cc74b0-kube-api-access-sqmnb\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:30:22.131953 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.131959    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b38d6234-7c3d-4fa2-a328-da5902cc74b0-tls-certs\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:30:22.132135 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.131967    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b38d6234-7c3d-4fa2-a328-da5902cc74b0-dshm\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:30:22.668941 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.668903    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5f7483b-b257-4119-93af-b95f58a7972c" path="/var/lib/kubelet/pods/f5f7483b-b257-4119-93af-b95f58a7972c/volumes"
Jul 13 22:30:22.940698 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.940623    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz" event={"ID":"b38d6234-7c3d-4fa2-a328-da5902cc74b0","Type":"ContainerDied","Data":"8b13ff38ca14ac7535f692a4d63f9b6ae0926e417805901fd27df37c30755f01"}
Jul 13 22:30:22.940698 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.940660    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"
Jul 13 22:30:22.940698 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.940668    2558 scope.go:117] "RemoveContainer" containerID="54f2bf037980b897f9b6c3c053c578a11f3c292bf077757f0e2c6ce8c0c2e3d0"
Jul 13 22:30:22.957167 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.957143    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"]
Jul 13 22:30:22.961128 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:22.961107    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-61b7b8/test-model-61b7b8-kserve-5449df4785-9gzkz"]
Jul 13 22:30:24.665868 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:24.665839    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b38d6234-7c3d-4fa2-a328-da5902cc74b0" path="/var/lib/kubelet/pods/b38d6234-7c3d-4fa2-a328-da5902cc74b0/volumes"
Jul 13 22:30:53.536470 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:30:53.536435    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:31:08.944920 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:31:08.944889    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:31:23.441289 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:31:23.441253    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:31:35.526851 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:31:35.526822    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:31:43.942031 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:31:43.942000    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:32:00.942059 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.942027    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:32:00.943412 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.943387    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"]
Jul 13 22:32:00.943728 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.943714    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5f7483b-b257-4119-93af-b95f58a7972c" containerName="main"
Jul 13 22:32:00.943777 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.943730    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5f7483b-b257-4119-93af-b95f58a7972c" containerName="main"
Jul 13 22:32:00.943777 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.943747    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b38d6234-7c3d-4fa2-a328-da5902cc74b0" containerName="main"
Jul 13 22:32:00.943777 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.943752    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b38d6234-7c3d-4fa2-a328-da5902cc74b0" containerName="main"
Jul 13 22:32:00.943877 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.943814    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b38d6234-7c3d-4fa2-a328-da5902cc74b0" containerName="main"
Jul 13 22:32:00.943877 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.943823    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5f7483b-b257-4119-93af-b95f58a7972c" containerName="main"
Jul 13 22:32:00.946956 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.946939    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:00.949933 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.949911    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0eaefd\"/\"kube-root-ca.crt\""
Jul 13 22:32:00.950091 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.949917    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0eaefd\"/\"default-dockercfg-snjpb\""
Jul 13 22:32:00.950091 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.950004    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0eaefd\"/\"openshift-service-ca.crt\""
Jul 13 22:32:00.950091 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.949918    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0eaefd\"/\"auth-test-model-0eaefd-kserve-self-signed-certs\""
Jul 13 22:32:00.955867 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:00.955845    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"]
Jul 13 22:32:01.032812 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.032785    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-dshm\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.032937 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.032815    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-tmp-dir\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.032937 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.032900    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-home\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.032937 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.032926    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjqb9\" (UniqueName: \"kubernetes.io/projected/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-kube-api-access-hjqb9\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.033082 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.033008    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-model-cache\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.033127 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.033108    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-tls-certs\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.133451 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.133428    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-model-cache\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.133595 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.133474    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-tls-certs\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.133595 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.133498    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-dshm\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.133595 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.133519    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-tmp-dir\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.133779 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.133600    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-home\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.133779 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.133619    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjqb9\" (UniqueName: \"kubernetes.io/projected/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-kube-api-access-hjqb9\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.133944 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.133924    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-model-cache\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.134000 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.133947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-tmp-dir\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.134037 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.134009    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-home\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.135734 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.135712    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-dshm\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.135989 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.135973    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-tls-certs\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.145266 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.145241    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjqb9\" (UniqueName: \"kubernetes.io/projected/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-kube-api-access-hjqb9\") pod \"auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") " pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.256845 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.256817    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:01.386203 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:01.386165    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"]
Jul 13 22:32:01.390769 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:32:01.390740    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a5a5dd7_16e2_4056_ad8e_f02ae680480a.slice/crio-ca79ebd84b3782b3affa807b1976cd50ebb704d6d8d8f57ad8da660a46acd1e8 WatchSource:0}: Error finding container ca79ebd84b3782b3affa807b1976cd50ebb704d6d8d8f57ad8da660a46acd1e8: Status 404 returned error can't find the container with id ca79ebd84b3782b3affa807b1976cd50ebb704d6d8d8f57ad8da660a46acd1e8
Jul 13 22:32:02.393517 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:02.393481    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4" event={"ID":"4a5a5dd7-16e2-4056-ad8e-f02ae680480a","Type":"ContainerStarted","Data":"472c7895635d01887acf3464c46135f0a7161935a44f545b933c23c4d9a7898b"}
Jul 13 22:32:02.393910 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:02.393532    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4" event={"ID":"4a5a5dd7-16e2-4056-ad8e-f02ae680480a","Type":"ContainerStarted","Data":"ca79ebd84b3782b3affa807b1976cd50ebb704d6d8d8f57ad8da660a46acd1e8"}
Jul 13 22:32:02.413456 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:02.413409    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4" podStartSLOduration=2.413394191 podStartE2EDuration="2.413394191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:32:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:32:02.411739326 +0000 UTC m=+2010.329715788" watchObservedRunningTime="2026-07-13 22:32:02.413394191 +0000 UTC m=+2010.331370654"
Jul 13 22:32:11.257430 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:11.257395    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:11.257818 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:11.257443    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:11.269224 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:11.269202    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:11.443213 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:11.443184    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:32:12.228739 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:12.228703    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:32:41.346247 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.346206    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"]
Jul 13 22:32:41.349590 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.349568    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.353279 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.353245    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4b5279\"/\"default-dockercfg-g2j4n\""
Jul 13 22:32:41.353279 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.353254    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4b5279\"/\"openshift-service-ca.crt\""
Jul 13 22:32:41.353279 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.353248    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4b5279\"/\"auth-test-model-4b5279-kserve-self-signed-certs\""
Jul 13 22:32:41.353279 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.353248    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4b5279\"/\"kube-root-ca.crt\""
Jul 13 22:32:41.359062 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.359038    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"]
Jul 13 22:32:41.456867 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.456835    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-home\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.457012 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.456890    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac82cff1-3618-4c9a-95fc-3790e367948b-tls-certs\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.457012 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.456953    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-dshm\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.457012 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.456998    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5f5l\" (UniqueName: \"kubernetes.io/projected/ac82cff1-3618-4c9a-95fc-3790e367948b-kube-api-access-k5f5l\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.457161 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.457023    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-tmp-dir\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.457161 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.457050    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-model-cache\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.558165 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.558138    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5f5l\" (UniqueName: \"kubernetes.io/projected/ac82cff1-3618-4c9a-95fc-3790e367948b-kube-api-access-k5f5l\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.558286 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.558168    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-tmp-dir\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.558286 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.558186    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-model-cache\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.558286 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.558221    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-home\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.558286 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.558260    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac82cff1-3618-4c9a-95fc-3790e367948b-tls-certs\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.558286 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.558284    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-dshm\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.558627 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.558607    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-model-cache\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.558676 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.558626    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-tmp-dir\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.558753 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.558736    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-home\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.560488 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.560473    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-dshm\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.560628 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.560613    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac82cff1-3618-4c9a-95fc-3790e367948b-tls-certs\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.566607 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.566589    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5f5l\" (UniqueName: \"kubernetes.io/projected/ac82cff1-3618-4c9a-95fc-3790e367948b-kube-api-access-k5f5l\") pod \"auth-test-model-4b5279-kserve-6955c45cf-4tbwn\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") " pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.659169 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.659106    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:41.781418 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:41.781394    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"]
Jul 13 22:32:41.783331 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:32:41.783296    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac82cff1_3618_4c9a_95fc_3790e367948b.slice/crio-f07982e4716d2efbe549d1a9da79607b8eac7f2c9cbd05f242a61c8071ccc69e WatchSource:0}: Error finding container f07982e4716d2efbe549d1a9da79607b8eac7f2c9cbd05f242a61c8071ccc69e: Status 404 returned error can't find the container with id f07982e4716d2efbe549d1a9da79607b8eac7f2c9cbd05f242a61c8071ccc69e
Jul 13 22:32:42.577828 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:42.577794    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn" event={"ID":"ac82cff1-3618-4c9a-95fc-3790e367948b","Type":"ContainerStarted","Data":"3c7507c520d0f2793bb8928f6c8b1e83b07d7534f66f874a5330201d96126e66"}
Jul 13 22:32:42.578159 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:42.577836    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn" event={"ID":"ac82cff1-3618-4c9a-95fc-3790e367948b","Type":"ContainerStarted","Data":"f07982e4716d2efbe549d1a9da79607b8eac7f2c9cbd05f242a61c8071ccc69e"}
Jul 13 22:32:42.594746 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:42.594703    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn" podStartSLOduration=1.5946895479999998 podStartE2EDuration="1.594689548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:32:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:32:42.593145576 +0000 UTC m=+2050.511122040" watchObservedRunningTime="2026-07-13 22:32:42.594689548 +0000 UTC m=+2050.512666012"
Jul 13 22:32:45.131231 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:45.131202    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:32:51.659214 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:51.659185    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:51.659214 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:51.659221    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:51.671365 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:51.671343    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:32:52.630773 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:32:52.630746    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:33:02.439276 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:33:02.439189    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:33:18.240696 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:33:18.240661    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:33:32.719090 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:33:32.719064    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:33:32.723213 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:33:32.723192    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:33:35.138312 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:33:35.138280    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:33:53.732672 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:33:53.732620    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:34:10.929158 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:10.929124    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:34:27.337130 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:27.337064    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:34:41.716573 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:41.716524    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"]
Jul 13 22:34:42.520624 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:42.520588    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"]
Jul 13 22:34:46.635826 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:46.635791    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:34:54.157340 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.157311    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"]
Jul 13 22:34:54.160292 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.160277    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.162924 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.162891    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0eaefd\"/\"sub-test-model-0eaefd-kserve-self-signed-certs\""
Jul 13 22:34:54.171674 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.171653    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"]
Jul 13 22:34:54.344624 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.344598    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-dshm\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.344788 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.344642    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f225c2-63ff-4763-a40d-d1b68c9a38c2-tls-certs\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.344788 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.344676    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-model-cache\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.344788 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.344692    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-tmp-dir\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.344788 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.344724    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-home\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.344788 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.344744    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdpfl\" (UniqueName: \"kubernetes.io/projected/70f225c2-63ff-4763-a40d-d1b68c9a38c2-kube-api-access-sdpfl\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.445982 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.445914    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f225c2-63ff-4763-a40d-d1b68c9a38c2-tls-certs\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.445982 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.445952    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-model-cache\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.445982 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.445970    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-tmp-dir\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.446190 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.446007    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-home\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.446190 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.446035    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdpfl\" (UniqueName: \"kubernetes.io/projected/70f225c2-63ff-4763-a40d-d1b68c9a38c2-kube-api-access-sdpfl\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.446190 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.446055    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-dshm\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.446341 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.446303    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-tmp-dir\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.446412 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.446392    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-model-cache\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.446468 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.446449    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-home\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.448282 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.448263    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-dshm\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.448423 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.448408    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f225c2-63ff-4763-a40d-d1b68c9a38c2-tls-certs\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.454886 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.454859    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdpfl\" (UniqueName: \"kubernetes.io/projected/70f225c2-63ff-4763-a40d-d1b68c9a38c2-kube-api-access-sdpfl\") pod \"sub-test-model-0eaefd-kserve-5bbc5d669-9x9db\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") " pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.469627 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.469610    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:34:54.598295 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:54.598268    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"]
Jul 13 22:34:54.600992 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:34:54.600966    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70f225c2_63ff_4763_a40d_d1b68c9a38c2.slice/crio-bf5ac671eb8d57757834cb3795228394f514b3444cfd78fd2692a713354dbbd5 WatchSource:0}: Error finding container bf5ac671eb8d57757834cb3795228394f514b3444cfd78fd2692a713354dbbd5: Status 404 returned error can't find the container with id bf5ac671eb8d57757834cb3795228394f514b3444cfd78fd2692a713354dbbd5
Jul 13 22:34:55.161350 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:55.161313    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db" event={"ID":"70f225c2-63ff-4763-a40d-d1b68c9a38c2","Type":"ContainerStarted","Data":"ee0f1885494355ddeee61b36e0c83abac824aa046c71eab0e12d616b59e24c7c"}
Jul 13 22:34:55.161350 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:55.161354    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db" event={"ID":"70f225c2-63ff-4763-a40d-d1b68c9a38c2","Type":"ContainerStarted","Data":"bf5ac671eb8d57757834cb3795228394f514b3444cfd78fd2692a713354dbbd5"}
Jul 13 22:34:55.179483 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:55.179440    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db" podStartSLOduration=1.179424758 podStartE2EDuration="1.179424758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:34:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:34:55.178238343 +0000 UTC m=+2183.096214808" watchObservedRunningTime="2026-07-13 22:34:55.179424758 +0000 UTC m=+2183.097401222"
Jul 13 22:34:56.736093 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:56.736053    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4" podUID="4a5a5dd7-16e2-4056-ad8e-f02ae680480a" containerName="main" containerID="cri-o://472c7895635d01887acf3464c46135f0a7161935a44f545b933c23c4d9a7898b" gracePeriod=45
Jul 13 22:34:56.984359 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:56.984337    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:34:57.168060 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.167985    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-home\") pod \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") "
Jul 13 22:34:57.168060 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.168046    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjqb9\" (UniqueName: \"kubernetes.io/projected/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-kube-api-access-hjqb9\") pod \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") "
Jul 13 22:34:57.168265 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.168067    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-dshm\") pod \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") "
Jul 13 22:34:57.168265 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.168164    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-model-cache\") pod \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") "
Jul 13 22:34:57.168265 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.168194    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-tls-certs\") pod \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") "
Jul 13 22:34:57.168265 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.168223    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-tmp-dir\") pod \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\" (UID: \"4a5a5dd7-16e2-4056-ad8e-f02ae680480a\") "
Jul 13 22:34:57.168265 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.168228    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-home" (OuterVolumeSpecName: "home") pod "4a5a5dd7-16e2-4056-ad8e-f02ae680480a" (UID: "4a5a5dd7-16e2-4056-ad8e-f02ae680480a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:34:57.168516 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.168421    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-model-cache" (OuterVolumeSpecName: "model-cache") pod "4a5a5dd7-16e2-4056-ad8e-f02ae680480a" (UID: "4a5a5dd7-16e2-4056-ad8e-f02ae680480a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:34:57.168516 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.168465    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-home\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:34:57.168516 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.168501    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4a5a5dd7-16e2-4056-ad8e-f02ae680480a" (UID: "4a5a5dd7-16e2-4056-ad8e-f02ae680480a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:34:57.170350 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.170325    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-dshm" (OuterVolumeSpecName: "dshm") pod "4a5a5dd7-16e2-4056-ad8e-f02ae680480a" (UID: "4a5a5dd7-16e2-4056-ad8e-f02ae680480a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:34:57.170469 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.170414    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4a5a5dd7-16e2-4056-ad8e-f02ae680480a" (UID: "4a5a5dd7-16e2-4056-ad8e-f02ae680480a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:34:57.170469 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.170425    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-kube-api-access-hjqb9" (OuterVolumeSpecName: "kube-api-access-hjqb9") pod "4a5a5dd7-16e2-4056-ad8e-f02ae680480a" (UID: "4a5a5dd7-16e2-4056-ad8e-f02ae680480a"). InnerVolumeSpecName "kube-api-access-hjqb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:34:57.172741 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.172717    2558 generic.go:356] "Generic (PLEG): container finished" podID="4a5a5dd7-16e2-4056-ad8e-f02ae680480a" containerID="472c7895635d01887acf3464c46135f0a7161935a44f545b933c23c4d9a7898b" exitCode=0
Jul 13 22:34:57.172861 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.172757    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"
Jul 13 22:34:57.172861 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.172776    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4" event={"ID":"4a5a5dd7-16e2-4056-ad8e-f02ae680480a","Type":"ContainerDied","Data":"472c7895635d01887acf3464c46135f0a7161935a44f545b933c23c4d9a7898b"}
Jul 13 22:34:57.172861 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.172841    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4" event={"ID":"4a5a5dd7-16e2-4056-ad8e-f02ae680480a","Type":"ContainerDied","Data":"ca79ebd84b3782b3affa807b1976cd50ebb704d6d8d8f57ad8da660a46acd1e8"}
Jul 13 22:34:57.173018 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.172864    2558 scope.go:117] "RemoveContainer" containerID="472c7895635d01887acf3464c46135f0a7161935a44f545b933c23c4d9a7898b"
Jul 13 22:34:57.188410 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.188393    2558 scope.go:117] "RemoveContainer" containerID="472c7895635d01887acf3464c46135f0a7161935a44f545b933c23c4d9a7898b"
Jul 13 22:34:57.188674 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:34:57.188642    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"472c7895635d01887acf3464c46135f0a7161935a44f545b933c23c4d9a7898b\": container with ID starting with 472c7895635d01887acf3464c46135f0a7161935a44f545b933c23c4d9a7898b not found: ID does not exist" containerID="472c7895635d01887acf3464c46135f0a7161935a44f545b933c23c4d9a7898b"
Jul 13 22:34:57.188735 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.188683    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"472c7895635d01887acf3464c46135f0a7161935a44f545b933c23c4d9a7898b"} err="failed to get container status \"472c7895635d01887acf3464c46135f0a7161935a44f545b933c23c4d9a7898b\": rpc error: code = NotFound desc = could not find container \"472c7895635d01887acf3464c46135f0a7161935a44f545b933c23c4d9a7898b\": container with ID starting with 472c7895635d01887acf3464c46135f0a7161935a44f545b933c23c4d9a7898b not found: ID does not exist"
Jul 13 22:34:57.198711 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.198687    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"]
Jul 13 22:34:57.201362 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.201340    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/auth-test-model-0eaefd-kserve-564bd6cd47-6fbv4"]
Jul 13 22:34:57.269331 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.269303    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-tmp-dir\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:34:57.269331 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.269326    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjqb9\" (UniqueName: \"kubernetes.io/projected/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-kube-api-access-hjqb9\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:34:57.269331 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.269335    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-dshm\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:34:57.269504 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.269344    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-model-cache\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:34:57.269504 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.269355    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a5a5dd7-16e2-4056-ad8e-f02ae680480a-tls-certs\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:34:57.540338 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.540304    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn" podUID="ac82cff1-3618-4c9a-95fc-3790e367948b" containerName="main" containerID="cri-o://3c7507c520d0f2793bb8928f6c8b1e83b07d7534f66f874a5330201d96126e66" gracePeriod=45
Jul 13 22:34:57.805956 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.805902    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:34:57.978092 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.977333    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac82cff1-3618-4c9a-95fc-3790e367948b-tls-certs\") pod \"ac82cff1-3618-4c9a-95fc-3790e367948b\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") "
Jul 13 22:34:57.978092 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.977397    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5f5l\" (UniqueName: \"kubernetes.io/projected/ac82cff1-3618-4c9a-95fc-3790e367948b-kube-api-access-k5f5l\") pod \"ac82cff1-3618-4c9a-95fc-3790e367948b\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") "
Jul 13 22:34:57.978092 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.977430    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-home\") pod \"ac82cff1-3618-4c9a-95fc-3790e367948b\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") "
Jul 13 22:34:57.978092 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.977473    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-model-cache\") pod \"ac82cff1-3618-4c9a-95fc-3790e367948b\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") "
Jul 13 22:34:57.978092 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.977523    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-tmp-dir\") pod \"ac82cff1-3618-4c9a-95fc-3790e367948b\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") "
Jul 13 22:34:57.978092 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.977569    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-dshm\") pod \"ac82cff1-3618-4c9a-95fc-3790e367948b\" (UID: \"ac82cff1-3618-4c9a-95fc-3790e367948b\") "
Jul 13 22:34:57.978092 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.977951    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-home" (OuterVolumeSpecName: "home") pod "ac82cff1-3618-4c9a-95fc-3790e367948b" (UID: "ac82cff1-3618-4c9a-95fc-3790e367948b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:34:57.978616 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.978360    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-model-cache" (OuterVolumeSpecName: "model-cache") pod "ac82cff1-3618-4c9a-95fc-3790e367948b" (UID: "ac82cff1-3618-4c9a-95fc-3790e367948b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:34:57.978616 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.978372    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ac82cff1-3618-4c9a-95fc-3790e367948b" (UID: "ac82cff1-3618-4c9a-95fc-3790e367948b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:34:57.988212 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.987946    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-dshm" (OuterVolumeSpecName: "dshm") pod "ac82cff1-3618-4c9a-95fc-3790e367948b" (UID: "ac82cff1-3618-4c9a-95fc-3790e367948b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:34:57.989689 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:57.989653    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac82cff1-3618-4c9a-95fc-3790e367948b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ac82cff1-3618-4c9a-95fc-3790e367948b" (UID: "ac82cff1-3618-4c9a-95fc-3790e367948b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:34:58.000967 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.000939    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac82cff1-3618-4c9a-95fc-3790e367948b-kube-api-access-k5f5l" (OuterVolumeSpecName: "kube-api-access-k5f5l") pod "ac82cff1-3618-4c9a-95fc-3790e367948b" (UID: "ac82cff1-3618-4c9a-95fc-3790e367948b"). InnerVolumeSpecName "kube-api-access-k5f5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:34:58.078645 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.078586    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-model-cache\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:34:58.078645 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.078610    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-tmp-dir\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:34:58.078645 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.078618    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-dshm\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:34:58.078645 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.078625    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac82cff1-3618-4c9a-95fc-3790e367948b-tls-certs\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:34:58.078645 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.078633    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5f5l\" (UniqueName: \"kubernetes.io/projected/ac82cff1-3618-4c9a-95fc-3790e367948b-kube-api-access-k5f5l\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:34:58.078645 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.078641    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac82cff1-3618-4c9a-95fc-3790e367948b-home\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:34:58.179464 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.179439    2558 generic.go:356] "Generic (PLEG): container finished" podID="ac82cff1-3618-4c9a-95fc-3790e367948b" containerID="3c7507c520d0f2793bb8928f6c8b1e83b07d7534f66f874a5330201d96126e66" exitCode=0
Jul 13 22:34:58.179606 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.179490    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"
Jul 13 22:34:58.179606 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.179520    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn" event={"ID":"ac82cff1-3618-4c9a-95fc-3790e367948b","Type":"ContainerDied","Data":"3c7507c520d0f2793bb8928f6c8b1e83b07d7534f66f874a5330201d96126e66"}
Jul 13 22:34:58.179606 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.179578    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn" event={"ID":"ac82cff1-3618-4c9a-95fc-3790e367948b","Type":"ContainerDied","Data":"f07982e4716d2efbe549d1a9da79607b8eac7f2c9cbd05f242a61c8071ccc69e"}
Jul 13 22:34:58.179606 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.179596    2558 scope.go:117] "RemoveContainer" containerID="3c7507c520d0f2793bb8928f6c8b1e83b07d7534f66f874a5330201d96126e66"
Jul 13 22:34:58.187903 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.187887    2558 scope.go:117] "RemoveContainer" containerID="3c7507c520d0f2793bb8928f6c8b1e83b07d7534f66f874a5330201d96126e66"
Jul 13 22:34:58.188136 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:34:58.188119    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c7507c520d0f2793bb8928f6c8b1e83b07d7534f66f874a5330201d96126e66\": container with ID starting with 3c7507c520d0f2793bb8928f6c8b1e83b07d7534f66f874a5330201d96126e66 not found: ID does not exist" containerID="3c7507c520d0f2793bb8928f6c8b1e83b07d7534f66f874a5330201d96126e66"
Jul 13 22:34:58.188194 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.188141    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c7507c520d0f2793bb8928f6c8b1e83b07d7534f66f874a5330201d96126e66"} err="failed to get container status \"3c7507c520d0f2793bb8928f6c8b1e83b07d7534f66f874a5330201d96126e66\": rpc error: code = NotFound desc = could not find container \"3c7507c520d0f2793bb8928f6c8b1e83b07d7534f66f874a5330201d96126e66\": container with ID starting with 3c7507c520d0f2793bb8928f6c8b1e83b07d7534f66f874a5330201d96126e66 not found: ID does not exist"
Jul 13 22:34:58.202470 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.202450    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"]
Jul 13 22:34:58.209310 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.209288    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/auth-test-model-4b5279-kserve-6955c45cf-4tbwn"]
Jul 13 22:34:58.666577 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.666532    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a5a5dd7-16e2-4056-ad8e-f02ae680480a" path="/var/lib/kubelet/pods/4a5a5dd7-16e2-4056-ad8e-f02ae680480a/volumes"
Jul 13 22:34:58.666942 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:34:58.666929    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac82cff1-3618-4c9a-95fc-3790e367948b" path="/var/lib/kubelet/pods/ac82cff1-3618-4c9a-95fc-3790e367948b/volumes"
Jul 13 22:35:04.470074 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:04.470030    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:35:04.470478 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:04.470335    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:35:04.482405 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:04.482383    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:35:05.222454 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:05.222423    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:35:06.732835 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:06.732804    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:35:23.140561 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.140506    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"]
Jul 13 22:35:23.141120 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.141021    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac82cff1-3618-4c9a-95fc-3790e367948b" containerName="main"
Jul 13 22:35:23.141120 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.141038    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac82cff1-3618-4c9a-95fc-3790e367948b" containerName="main"
Jul 13 22:35:23.141120 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.141101    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a5a5dd7-16e2-4056-ad8e-f02ae680480a" containerName="main"
Jul 13 22:35:23.141120 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.141109    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a5a5dd7-16e2-4056-ad8e-f02ae680480a" containerName="main"
Jul 13 22:35:23.141353 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.141184    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a5a5dd7-16e2-4056-ad8e-f02ae680480a" containerName="main"
Jul 13 22:35:23.141353 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.141200    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac82cff1-3618-4c9a-95fc-3790e367948b" containerName="main"
Jul 13 22:35:23.144713 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.144692    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.147211 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.147192    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4b5279\"/\"kube-root-ca.crt\""
Jul 13 22:35:23.148194 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.148176    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4b5279\"/\"default-dockercfg-g2j4n\""
Jul 13 22:35:23.148194 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.148184    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4b5279\"/\"openshift-service-ca.crt\""
Jul 13 22:35:23.148348 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.148282    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4b5279\"/\"sub-test-model-4b5279-kserve-self-signed-certs\""
Jul 13 22:35:23.153224 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.153201    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"]
Jul 13 22:35:23.172635 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.172615    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cc581f3-5c97-44de-90f8-b5aaa704978b-tls-certs\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.172735 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.172642    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fxtl\" (UniqueName: \"kubernetes.io/projected/2cc581f3-5c97-44de-90f8-b5aaa704978b-kube-api-access-2fxtl\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.172735 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.172677    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-model-cache\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.172804 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.172753    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-tmp-dir\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.172804 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.172795    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-home\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.172875 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.172852    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-dshm\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.273907 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.273881    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cc581f3-5c97-44de-90f8-b5aaa704978b-tls-certs\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.274005 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.273911    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fxtl\" (UniqueName: \"kubernetes.io/projected/2cc581f3-5c97-44de-90f8-b5aaa704978b-kube-api-access-2fxtl\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.274005 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.273931    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-model-cache\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.274005 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.273972    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-tmp-dir\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.274106 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.274018    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-home\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.274106 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.274065    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-dshm\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.274348 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.274328    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-model-cache\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.274418 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.274404    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-tmp-dir\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.274476 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.274427    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-home\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.276238 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.276214    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-dshm\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.276474 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.276457    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cc581f3-5c97-44de-90f8-b5aaa704978b-tls-certs\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.282446 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.282427    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fxtl\" (UniqueName: \"kubernetes.io/projected/2cc581f3-5c97-44de-90f8-b5aaa704978b-kube-api-access-2fxtl\") pod \"sub-test-model-4b5279-kserve-5f475898f-jt6tz\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") " pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.455056 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.454980    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:23.583090 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:23.583064    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"]
Jul 13 22:35:23.585502 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:35:23.585476    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2cc581f3_5c97_44de_90f8_b5aaa704978b.slice/crio-6bb425bfff08851a604192b9080e7a7f7af2cd17a1ef5ee44e2b51119f077efb WatchSource:0}: Error finding container 6bb425bfff08851a604192b9080e7a7f7af2cd17a1ef5ee44e2b51119f077efb: Status 404 returned error can't find the container with id 6bb425bfff08851a604192b9080e7a7f7af2cd17a1ef5ee44e2b51119f077efb
Jul 13 22:35:24.298407 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:24.298372    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz" event={"ID":"2cc581f3-5c97-44de-90f8-b5aaa704978b","Type":"ContainerStarted","Data":"975d846e936a0c3cfce7c2eb2cb52283ad0f821cd195973a0fffd9d4fe4bdaac"}
Jul 13 22:35:24.298407 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:24.298410    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz" event={"ID":"2cc581f3-5c97-44de-90f8-b5aaa704978b","Type":"ContainerStarted","Data":"6bb425bfff08851a604192b9080e7a7f7af2cd17a1ef5ee44e2b51119f077efb"}
Jul 13 22:35:24.314510 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:24.314468    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz" podStartSLOduration=1.314454125 podStartE2EDuration="1.314454125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:35:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:35:24.313873277 +0000 UTC m=+2212.231849743" watchObservedRunningTime="2026-07-13 22:35:24.314454125 +0000 UTC m=+2212.232430590"
Jul 13 22:35:27.738315 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:27.738279    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:35:33.455861 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:33.455824    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:33.456256 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:33.455879    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:33.468231 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:33.468210    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:34.352556 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:34.352506    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:35:45.927515 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:35:45.927482    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:36:01.256483 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:01.256411    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"]
Jul 13 22:36:02.303973 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:02.303937    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"]
Jul 13 22:36:06.729144 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:06.729102    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:36:14.320043 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.320006    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"]
Jul 13 22:36:14.323312 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.323292    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.330633 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.330614    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0eaefd\"/\"rate-test-model-0eaefd-kserve-self-signed-certs\""
Jul 13 22:36:14.339379 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.339360    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"]
Jul 13 22:36:14.405351 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.405328    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f756c06a-05ba-4bbc-91e5-6c5627463662-tls-certs\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.405447 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.405371    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-tmp-dir\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.405447 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.405392    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-home\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.405447 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.405415    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-dshm\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.405575 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.405476    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-model-cache\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.405575 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.405549    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq4zq\" (UniqueName: \"kubernetes.io/projected/f756c06a-05ba-4bbc-91e5-6c5627463662-kube-api-access-zq4zq\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.506232 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.506211    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f756c06a-05ba-4bbc-91e5-6c5627463662-tls-certs\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.506342 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.506251    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-tmp-dir\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.506342 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.506268    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-home\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.506342 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.506286    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-dshm\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.506510 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.506391    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-model-cache\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.506510 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.506451    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq4zq\" (UniqueName: \"kubernetes.io/projected/f756c06a-05ba-4bbc-91e5-6c5627463662-kube-api-access-zq4zq\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.506657 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.506636    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-home\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.506717 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.506699    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-tmp-dir\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.506819 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.506798    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-model-cache\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.508527 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.508509    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-dshm\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.508730 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.508713    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f756c06a-05ba-4bbc-91e5-6c5627463662-tls-certs\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.515292 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.515274    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq4zq\" (UniqueName: \"kubernetes.io/projected/f756c06a-05ba-4bbc-91e5-6c5627463662-kube-api-access-zq4zq\") pod \"rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") " pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.632711 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.632647    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:14.762434 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:14.762413    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"]
Jul 13 22:36:15.523100 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:15.523064    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk" event={"ID":"f756c06a-05ba-4bbc-91e5-6c5627463662","Type":"ContainerStarted","Data":"883812f0604b319cd13108ba923e2d9e3f4d980e503d850f0df8f01ab522e08b"}
Jul 13 22:36:15.523439 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:15.523112    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk" event={"ID":"f756c06a-05ba-4bbc-91e5-6c5627463662","Type":"ContainerStarted","Data":"f0c3c4793d8a9914760cc7d9d2c3d1010c361178b436c4f897d4122f371eb8bc"}
Jul 13 22:36:15.542682 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:15.542638    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk" podStartSLOduration=1.542624706 podStartE2EDuration="1.542624706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:36:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:36:15.540925532 +0000 UTC m=+2263.458902007" watchObservedRunningTime="2026-07-13 22:36:15.542624706 +0000 UTC m=+2263.460601161"
Jul 13 22:36:16.275713 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.275675    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db" podUID="70f225c2-63ff-4763-a40d-d1b68c9a38c2" containerName="main" containerID="cri-o://ee0f1885494355ddeee61b36e0c83abac824aa046c71eab0e12d616b59e24c7c" gracePeriod=45
Jul 13 22:36:16.526674 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.526613    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:36:16.528750 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.528722    2558 generic.go:356] "Generic (PLEG): container finished" podID="70f225c2-63ff-4763-a40d-d1b68c9a38c2" containerID="ee0f1885494355ddeee61b36e0c83abac824aa046c71eab0e12d616b59e24c7c" exitCode=0
Jul 13 22:36:16.528874 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.528756    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"
Jul 13 22:36:16.528874 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.528802    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db" event={"ID":"70f225c2-63ff-4763-a40d-d1b68c9a38c2","Type":"ContainerDied","Data":"ee0f1885494355ddeee61b36e0c83abac824aa046c71eab0e12d616b59e24c7c"}
Jul 13 22:36:16.528874 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.528851    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db" event={"ID":"70f225c2-63ff-4763-a40d-d1b68c9a38c2","Type":"ContainerDied","Data":"bf5ac671eb8d57757834cb3795228394f514b3444cfd78fd2692a713354dbbd5"}
Jul 13 22:36:16.528874 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.528874    2558 scope.go:117] "RemoveContainer" containerID="ee0f1885494355ddeee61b36e0c83abac824aa046c71eab0e12d616b59e24c7c"
Jul 13 22:36:16.536638 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.536619    2558 scope.go:117] "RemoveContainer" containerID="ee0f1885494355ddeee61b36e0c83abac824aa046c71eab0e12d616b59e24c7c"
Jul 13 22:36:16.536886 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:36:16.536867    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee0f1885494355ddeee61b36e0c83abac824aa046c71eab0e12d616b59e24c7c\": container with ID starting with ee0f1885494355ddeee61b36e0c83abac824aa046c71eab0e12d616b59e24c7c not found: ID does not exist" containerID="ee0f1885494355ddeee61b36e0c83abac824aa046c71eab0e12d616b59e24c7c"
Jul 13 22:36:16.536944 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.536892    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee0f1885494355ddeee61b36e0c83abac824aa046c71eab0e12d616b59e24c7c"} err="failed to get container status \"ee0f1885494355ddeee61b36e0c83abac824aa046c71eab0e12d616b59e24c7c\": rpc error: code = NotFound desc = could not find container \"ee0f1885494355ddeee61b36e0c83abac824aa046c71eab0e12d616b59e24c7c\": container with ID starting with ee0f1885494355ddeee61b36e0c83abac824aa046c71eab0e12d616b59e24c7c not found: ID does not exist"
Jul 13 22:36:16.627940 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.627915    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f225c2-63ff-4763-a40d-d1b68c9a38c2-tls-certs\") pod \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") "
Jul 13 22:36:16.628068 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.627950    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-home\") pod \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") "
Jul 13 22:36:16.628068 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.627972    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdpfl\" (UniqueName: \"kubernetes.io/projected/70f225c2-63ff-4763-a40d-d1b68c9a38c2-kube-api-access-sdpfl\") pod \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") "
Jul 13 22:36:16.628068 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.628002    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-tmp-dir\") pod \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") "
Jul 13 22:36:16.628068 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.628028    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-dshm\") pod \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") "
Jul 13 22:36:16.628068 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.628068    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-model-cache\") pod \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\" (UID: \"70f225c2-63ff-4763-a40d-d1b68c9a38c2\") "
Jul 13 22:36:16.628270 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.628222    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-home" (OuterVolumeSpecName: "home") pod "70f225c2-63ff-4763-a40d-d1b68c9a38c2" (UID: "70f225c2-63ff-4763-a40d-d1b68c9a38c2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:36:16.628360 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.628322    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "70f225c2-63ff-4763-a40d-d1b68c9a38c2" (UID: "70f225c2-63ff-4763-a40d-d1b68c9a38c2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:36:16.628428 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.628391    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-home\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:36:16.628477 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.628407    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-model-cache" (OuterVolumeSpecName: "model-cache") pod "70f225c2-63ff-4763-a40d-d1b68c9a38c2" (UID: "70f225c2-63ff-4763-a40d-d1b68c9a38c2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:36:16.630063 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.630036    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70f225c2-63ff-4763-a40d-d1b68c9a38c2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "70f225c2-63ff-4763-a40d-d1b68c9a38c2" (UID: "70f225c2-63ff-4763-a40d-d1b68c9a38c2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:36:16.630156 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.630062    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-dshm" (OuterVolumeSpecName: "dshm") pod "70f225c2-63ff-4763-a40d-d1b68c9a38c2" (UID: "70f225c2-63ff-4763-a40d-d1b68c9a38c2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:36:16.630156 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.630135    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70f225c2-63ff-4763-a40d-d1b68c9a38c2-kube-api-access-sdpfl" (OuterVolumeSpecName: "kube-api-access-sdpfl") pod "70f225c2-63ff-4763-a40d-d1b68c9a38c2" (UID: "70f225c2-63ff-4763-a40d-d1b68c9a38c2"). InnerVolumeSpecName "kube-api-access-sdpfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:36:16.729133 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.729107    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70f225c2-63ff-4763-a40d-d1b68c9a38c2-tls-certs\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:36:16.729133 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.729130    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdpfl\" (UniqueName: \"kubernetes.io/projected/70f225c2-63ff-4763-a40d-d1b68c9a38c2-kube-api-access-sdpfl\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:36:16.729312 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.729146    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-tmp-dir\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:36:16.729312 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.729158    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-dshm\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:36:16.729312 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.729166    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70f225c2-63ff-4763-a40d-d1b68c9a38c2-model-cache\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:36:16.843500 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.843427    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"]
Jul 13 22:36:16.847070 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:16.847051    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/sub-test-model-0eaefd-kserve-5bbc5d669-9x9db"]
Jul 13 22:36:17.323228 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.323168    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz" podUID="2cc581f3-5c97-44de-90f8-b5aaa704978b" containerName="main" containerID="cri-o://975d846e936a0c3cfce7c2eb2cb52283ad0f821cd195973a0fffd9d4fe4bdaac" gracePeriod=45
Jul 13 22:36:17.535658 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.535631    2558 generic.go:356] "Generic (PLEG): container finished" podID="2cc581f3-5c97-44de-90f8-b5aaa704978b" containerID="975d846e936a0c3cfce7c2eb2cb52283ad0f821cd195973a0fffd9d4fe4bdaac" exitCode=0
Jul 13 22:36:17.535984 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.535698    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz" event={"ID":"2cc581f3-5c97-44de-90f8-b5aaa704978b","Type":"ContainerDied","Data":"975d846e936a0c3cfce7c2eb2cb52283ad0f821cd195973a0fffd9d4fe4bdaac"}
Jul 13 22:36:17.573221 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.573197    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:36:17.636077 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.636007    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-model-cache\") pod \"2cc581f3-5c97-44de-90f8-b5aaa704978b\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") "
Jul 13 22:36:17.636077 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.636040    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-tmp-dir\") pod \"2cc581f3-5c97-44de-90f8-b5aaa704978b\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") "
Jul 13 22:36:17.636077 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.636058    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-dshm\") pod \"2cc581f3-5c97-44de-90f8-b5aaa704978b\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") "
Jul 13 22:36:17.636295 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.636102    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fxtl\" (UniqueName: \"kubernetes.io/projected/2cc581f3-5c97-44de-90f8-b5aaa704978b-kube-api-access-2fxtl\") pod \"2cc581f3-5c97-44de-90f8-b5aaa704978b\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") "
Jul 13 22:36:17.636295 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.636171    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cc581f3-5c97-44de-90f8-b5aaa704978b-tls-certs\") pod \"2cc581f3-5c97-44de-90f8-b5aaa704978b\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") "
Jul 13 22:36:17.636295 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.636212    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-home\") pod \"2cc581f3-5c97-44de-90f8-b5aaa704978b\" (UID: \"2cc581f3-5c97-44de-90f8-b5aaa704978b\") "
Jul 13 22:36:17.636445 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.636293    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-model-cache" (OuterVolumeSpecName: "model-cache") pod "2cc581f3-5c97-44de-90f8-b5aaa704978b" (UID: "2cc581f3-5c97-44de-90f8-b5aaa704978b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:36:17.636445 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.636341    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2cc581f3-5c97-44de-90f8-b5aaa704978b" (UID: "2cc581f3-5c97-44de-90f8-b5aaa704978b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:36:17.636570 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.636478    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-home" (OuterVolumeSpecName: "home") pod "2cc581f3-5c97-44de-90f8-b5aaa704978b" (UID: "2cc581f3-5c97-44de-90f8-b5aaa704978b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:36:17.636570 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.636497    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-model-cache\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:36:17.636570 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.636512    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-tmp-dir\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:36:17.638254 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.638228    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cc581f3-5c97-44de-90f8-b5aaa704978b-kube-api-access-2fxtl" (OuterVolumeSpecName: "kube-api-access-2fxtl") pod "2cc581f3-5c97-44de-90f8-b5aaa704978b" (UID: "2cc581f3-5c97-44de-90f8-b5aaa704978b"). InnerVolumeSpecName "kube-api-access-2fxtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:36:17.638254 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.638241    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-dshm" (OuterVolumeSpecName: "dshm") pod "2cc581f3-5c97-44de-90f8-b5aaa704978b" (UID: "2cc581f3-5c97-44de-90f8-b5aaa704978b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:36:17.638369 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.638265    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cc581f3-5c97-44de-90f8-b5aaa704978b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2cc581f3-5c97-44de-90f8-b5aaa704978b" (UID: "2cc581f3-5c97-44de-90f8-b5aaa704978b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:36:17.736987 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.736963    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-home\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:36:17.736987 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.736985    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cc581f3-5c97-44de-90f8-b5aaa704978b-dshm\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:36:17.737135 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.736994    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fxtl\" (UniqueName: \"kubernetes.io/projected/2cc581f3-5c97-44de-90f8-b5aaa704978b-kube-api-access-2fxtl\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:36:17.737135 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:17.737004    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cc581f3-5c97-44de-90f8-b5aaa704978b-tls-certs\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:36:18.542743 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:18.542717    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"
Jul 13 22:36:18.543132 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:18.542766    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz" event={"ID":"2cc581f3-5c97-44de-90f8-b5aaa704978b","Type":"ContainerDied","Data":"6bb425bfff08851a604192b9080e7a7f7af2cd17a1ef5ee44e2b51119f077efb"}
Jul 13 22:36:18.543132 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:18.542816    2558 scope.go:117] "RemoveContainer" containerID="975d846e936a0c3cfce7c2eb2cb52283ad0f821cd195973a0fffd9d4fe4bdaac"
Jul 13 22:36:18.563046 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:18.562922    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"]
Jul 13 22:36:18.564749 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:18.564732    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/sub-test-model-4b5279-kserve-5f475898f-jt6tz"]
Jul 13 22:36:18.665669 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:18.665643    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cc581f3-5c97-44de-90f8-b5aaa704978b" path="/var/lib/kubelet/pods/2cc581f3-5c97-44de-90f8-b5aaa704978b/volumes"
Jul 13 22:36:18.666028 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:18.666014    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70f225c2-63ff-4763-a40d-d1b68c9a38c2" path="/var/lib/kubelet/pods/70f225c2-63ff-4763-a40d-d1b68c9a38c2/volumes"
Jul 13 22:36:21.139528 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:21.139490    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:36:24.633397 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:24.633366    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:24.633808 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:24.633415    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:24.645252 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:24.645231    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:25.584023 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:25.583995    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:36:59.339782 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.339743    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"]
Jul 13 22:36:59.340296 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.340253    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cc581f3-5c97-44de-90f8-b5aaa704978b" containerName="main"
Jul 13 22:36:59.340296 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.340279    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cc581f3-5c97-44de-90f8-b5aaa704978b" containerName="main"
Jul 13 22:36:59.340296 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.340292    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70f225c2-63ff-4763-a40d-d1b68c9a38c2" containerName="main"
Jul 13 22:36:59.340458 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.340300    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="70f225c2-63ff-4763-a40d-d1b68c9a38c2" containerName="main"
Jul 13 22:36:59.340458 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.340385    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cc581f3-5c97-44de-90f8-b5aaa704978b" containerName="main"
Jul 13 22:36:59.340458 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.340400    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="70f225c2-63ff-4763-a40d-d1b68c9a38c2" containerName="main"
Jul 13 22:36:59.343493 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.343471    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.347069 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.347049    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4b5279\"/\"default-dockercfg-g2j4n\""
Jul 13 22:36:59.347069 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.347060    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4b5279\"/\"rate-test-model-4b5279-kserve-self-signed-certs\""
Jul 13 22:36:59.347221 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.347140    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4b5279\"/\"kube-root-ca.crt\""
Jul 13 22:36:59.347221 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.347140    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4b5279\"/\"openshift-service-ca.crt\""
Jul 13 22:36:59.352888 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.352867    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"]
Jul 13 22:36:59.368653 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.368633    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-home\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.368743 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.368664    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-model-cache\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.368743 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.368686    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-tmp-dir\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.368743 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.368710    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk9h7\" (UniqueName: \"kubernetes.io/projected/26e05792-d6c7-4e41-92ef-0bb340d88078-kube-api-access-zk9h7\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.368743 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.368737    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-dshm\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.368872 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.368785    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26e05792-d6c7-4e41-92ef-0bb340d88078-tls-certs\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.470083 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.470058    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-home\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.470177 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.470092    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-model-cache\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.470177 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.470119    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-tmp-dir\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.470177 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.470154    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk9h7\" (UniqueName: \"kubernetes.io/projected/26e05792-d6c7-4e41-92ef-0bb340d88078-kube-api-access-zk9h7\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.470338 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.470204    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-dshm\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.470338 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.470228    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26e05792-d6c7-4e41-92ef-0bb340d88078-tls-certs\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.470500 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.470480    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-model-cache\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.470588 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.470499    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-home\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.470588 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.470563    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-tmp-dir\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.472399 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.472382    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-dshm\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.472779 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.472759    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26e05792-d6c7-4e41-92ef-0bb340d88078-tls-certs\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.478393 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.478372    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk9h7\" (UniqueName: \"kubernetes.io/projected/26e05792-d6c7-4e41-92ef-0bb340d88078-kube-api-access-zk9h7\") pod \"rate-test-model-4b5279-kserve-5cbbddff75-lgwzz\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") " pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.653107 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.653038    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:36:59.781064 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:36:59.781039    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"]
Jul 13 22:36:59.783427 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:36:59.783400    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26e05792_d6c7_4e41_92ef_0bb340d88078.slice/crio-f1a3a13d4120e694197a85b33c6a5611c08793162c710763badb787b4eba3c16 WatchSource:0}: Error finding container f1a3a13d4120e694197a85b33c6a5611c08793162c710763badb787b4eba3c16: Status 404 returned error can't find the container with id f1a3a13d4120e694197a85b33c6a5611c08793162c710763badb787b4eba3c16
Jul 13 22:37:00.732265 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:37:00.732233    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz" event={"ID":"26e05792-d6c7-4e41-92ef-0bb340d88078","Type":"ContainerStarted","Data":"e39361cd37691433d9c6a1d97ddb650c019c6656883c76b77f4f8583c33497dc"}
Jul 13 22:37:00.732265 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:37:00.732269    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz" event={"ID":"26e05792-d6c7-4e41-92ef-0bb340d88078","Type":"ContainerStarted","Data":"f1a3a13d4120e694197a85b33c6a5611c08793162c710763badb787b4eba3c16"}
Jul 13 22:37:03.845634 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:37:03.845583    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz" podStartSLOduration=4.845566573 podStartE2EDuration="4.845566573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:36:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:37:00.748943922 +0000 UTC m=+2308.666920388" watchObservedRunningTime="2026-07-13 22:37:03.845566573 +0000 UTC m=+2311.763543036"
Jul 13 22:37:03.846207 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:37:03.846189    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:37:09.653695 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:37:09.653662    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:37:09.654112 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:37:09.653777    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:37:09.665923 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:37:09.665898    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:37:09.781301 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:37:09.781276    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:37:25.337485 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:37:25.337411    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:37:42.670408 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:37:42.670372    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:37:47.983181 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:37:47.983149    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"]
Jul 13 22:37:49.012017 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:37:49.011986    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"]
Jul 13 22:38:01.140074 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:01.140046    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:38:03.002840 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.002785    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk" podUID="f756c06a-05ba-4bbc-91e5-6c5627463662" containerName="main" containerID="cri-o://883812f0604b319cd13108ba923e2d9e3f4d980e503d850f0df8f01ab522e08b" gracePeriod=45
Jul 13 22:38:03.254624 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.254574    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:38:03.370869 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.370838    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq4zq\" (UniqueName: \"kubernetes.io/projected/f756c06a-05ba-4bbc-91e5-6c5627463662-kube-api-access-zq4zq\") pod \"f756c06a-05ba-4bbc-91e5-6c5627463662\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") "
Jul 13 22:38:03.371034 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.370889    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f756c06a-05ba-4bbc-91e5-6c5627463662-tls-certs\") pod \"f756c06a-05ba-4bbc-91e5-6c5627463662\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") "
Jul 13 22:38:03.371034 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.370920    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-tmp-dir\") pod \"f756c06a-05ba-4bbc-91e5-6c5627463662\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") "
Jul 13 22:38:03.371194 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.371058    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-model-cache\") pod \"f756c06a-05ba-4bbc-91e5-6c5627463662\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") "
Jul 13 22:38:03.371194 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.371096    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-dshm\") pod \"f756c06a-05ba-4bbc-91e5-6c5627463662\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") "
Jul 13 22:38:03.371194 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.371129    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-home\") pod \"f756c06a-05ba-4bbc-91e5-6c5627463662\" (UID: \"f756c06a-05ba-4bbc-91e5-6c5627463662\") "
Jul 13 22:38:03.371194 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.371164    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f756c06a-05ba-4bbc-91e5-6c5627463662" (UID: "f756c06a-05ba-4bbc-91e5-6c5627463662"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:38:03.371389 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.371344    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-model-cache" (OuterVolumeSpecName: "model-cache") pod "f756c06a-05ba-4bbc-91e5-6c5627463662" (UID: "f756c06a-05ba-4bbc-91e5-6c5627463662"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:38:03.371439 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.371411    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-home" (OuterVolumeSpecName: "home") pod "f756c06a-05ba-4bbc-91e5-6c5627463662" (UID: "f756c06a-05ba-4bbc-91e5-6c5627463662"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:38:03.371495 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.371453    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-model-cache\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:38:03.371495 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.371486    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-tmp-dir\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:38:03.373082 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.373060    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-dshm" (OuterVolumeSpecName: "dshm") pod "f756c06a-05ba-4bbc-91e5-6c5627463662" (UID: "f756c06a-05ba-4bbc-91e5-6c5627463662"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:38:03.373439 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.373418    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f756c06a-05ba-4bbc-91e5-6c5627463662-kube-api-access-zq4zq" (OuterVolumeSpecName: "kube-api-access-zq4zq") pod "f756c06a-05ba-4bbc-91e5-6c5627463662" (UID: "f756c06a-05ba-4bbc-91e5-6c5627463662"). InnerVolumeSpecName "kube-api-access-zq4zq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:38:03.373502 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.373466    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f756c06a-05ba-4bbc-91e5-6c5627463662-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f756c06a-05ba-4bbc-91e5-6c5627463662" (UID: "f756c06a-05ba-4bbc-91e5-6c5627463662"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:38:03.472795 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.472770    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq4zq\" (UniqueName: \"kubernetes.io/projected/f756c06a-05ba-4bbc-91e5-6c5627463662-kube-api-access-zq4zq\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:38:03.472795 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.472790    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f756c06a-05ba-4bbc-91e5-6c5627463662-tls-certs\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:38:03.472795 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.472799    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-dshm\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:38:03.472992 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:03.472807    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f756c06a-05ba-4bbc-91e5-6c5627463662-home\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:38:04.017155 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.017127    2558 generic.go:356] "Generic (PLEG): container finished" podID="f756c06a-05ba-4bbc-91e5-6c5627463662" containerID="883812f0604b319cd13108ba923e2d9e3f4d980e503d850f0df8f01ab522e08b" exitCode=0
Jul 13 22:38:04.017507 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.017166    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"
Jul 13 22:38:04.017507 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.017169    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk" event={"ID":"f756c06a-05ba-4bbc-91e5-6c5627463662","Type":"ContainerDied","Data":"883812f0604b319cd13108ba923e2d9e3f4d980e503d850f0df8f01ab522e08b"}
Jul 13 22:38:04.017507 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.017199    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk" event={"ID":"f756c06a-05ba-4bbc-91e5-6c5627463662","Type":"ContainerDied","Data":"f0c3c4793d8a9914760cc7d9d2c3d1010c361178b436c4f897d4122f371eb8bc"}
Jul 13 22:38:04.017507 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.017214    2558 scope.go:117] "RemoveContainer" containerID="883812f0604b319cd13108ba923e2d9e3f4d980e503d850f0df8f01ab522e08b"
Jul 13 22:38:04.026745 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.026725    2558 scope.go:117] "RemoveContainer" containerID="883812f0604b319cd13108ba923e2d9e3f4d980e503d850f0df8f01ab522e08b"
Jul 13 22:38:04.027000 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:38:04.026974    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"883812f0604b319cd13108ba923e2d9e3f4d980e503d850f0df8f01ab522e08b\": container with ID starting with 883812f0604b319cd13108ba923e2d9e3f4d980e503d850f0df8f01ab522e08b not found: ID does not exist" containerID="883812f0604b319cd13108ba923e2d9e3f4d980e503d850f0df8f01ab522e08b"
Jul 13 22:38:04.027097 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.027005    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"883812f0604b319cd13108ba923e2d9e3f4d980e503d850f0df8f01ab522e08b"} err="failed to get container status \"883812f0604b319cd13108ba923e2d9e3f4d980e503d850f0df8f01ab522e08b\": rpc error: code = NotFound desc = could not find container \"883812f0604b319cd13108ba923e2d9e3f4d980e503d850f0df8f01ab522e08b\": container with ID starting with 883812f0604b319cd13108ba923e2d9e3f4d980e503d850f0df8f01ab522e08b not found: ID does not exist"
Jul 13 22:38:04.039727 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.039700    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"]
Jul 13 22:38:04.041127 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.041097    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz" podUID="26e05792-d6c7-4e41-92ef-0bb340d88078" containerName="main" containerID="cri-o://e39361cd37691433d9c6a1d97ddb650c019c6656883c76b77f4f8583c33497dc" gracePeriod=45
Jul 13 22:38:04.049035 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.049014    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0eaefd/rate-test-model-0eaefd-kserve-544bfd9c87-pkhbk"]
Jul 13 22:38:04.283852 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.283832    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:38:04.381054 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.381026    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk9h7\" (UniqueName: \"kubernetes.io/projected/26e05792-d6c7-4e41-92ef-0bb340d88078-kube-api-access-zk9h7\") pod \"26e05792-d6c7-4e41-92ef-0bb340d88078\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") "
Jul 13 22:38:04.381193 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.381062    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-model-cache\") pod \"26e05792-d6c7-4e41-92ef-0bb340d88078\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") "
Jul 13 22:38:04.381193 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.381081    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-tmp-dir\") pod \"26e05792-d6c7-4e41-92ef-0bb340d88078\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") "
Jul 13 22:38:04.381193 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.381103    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26e05792-d6c7-4e41-92ef-0bb340d88078-tls-certs\") pod \"26e05792-d6c7-4e41-92ef-0bb340d88078\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") "
Jul 13 22:38:04.381193 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.381144    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-dshm\") pod \"26e05792-d6c7-4e41-92ef-0bb340d88078\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") "
Jul 13 22:38:04.381395 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.381206    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-home\") pod \"26e05792-d6c7-4e41-92ef-0bb340d88078\" (UID: \"26e05792-d6c7-4e41-92ef-0bb340d88078\") "
Jul 13 22:38:04.381395 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.381334    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-model-cache" (OuterVolumeSpecName: "model-cache") pod "26e05792-d6c7-4e41-92ef-0bb340d88078" (UID: "26e05792-d6c7-4e41-92ef-0bb340d88078"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:38:04.381395 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.381355    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "26e05792-d6c7-4e41-92ef-0bb340d88078" (UID: "26e05792-d6c7-4e41-92ef-0bb340d88078"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:38:04.381623 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.381508    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-home" (OuterVolumeSpecName: "home") pod "26e05792-d6c7-4e41-92ef-0bb340d88078" (UID: "26e05792-d6c7-4e41-92ef-0bb340d88078"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:38:04.381623 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.381607    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-model-cache\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:38:04.381701 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.381627    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-tmp-dir\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:38:04.381701 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.381640    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-home\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:38:04.383402 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.383372    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-dshm" (OuterVolumeSpecName: "dshm") pod "26e05792-d6c7-4e41-92ef-0bb340d88078" (UID: "26e05792-d6c7-4e41-92ef-0bb340d88078"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:38:04.383690 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.383667    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26e05792-d6c7-4e41-92ef-0bb340d88078-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "26e05792-d6c7-4e41-92ef-0bb340d88078" (UID: "26e05792-d6c7-4e41-92ef-0bb340d88078"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:38:04.383790 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.383672    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26e05792-d6c7-4e41-92ef-0bb340d88078-kube-api-access-zk9h7" (OuterVolumeSpecName: "kube-api-access-zk9h7") pod "26e05792-d6c7-4e41-92ef-0bb340d88078" (UID: "26e05792-d6c7-4e41-92ef-0bb340d88078"). InnerVolumeSpecName "kube-api-access-zk9h7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:38:04.482346 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.482323    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26e05792-d6c7-4e41-92ef-0bb340d88078-dshm\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:38:04.482346 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.482343    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk9h7\" (UniqueName: \"kubernetes.io/projected/26e05792-d6c7-4e41-92ef-0bb340d88078-kube-api-access-zk9h7\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:38:04.482478 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.482354    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26e05792-d6c7-4e41-92ef-0bb340d88078-tls-certs\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:38:04.655988 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.655921    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7556-140665183697776"]
Jul 13 22:38:04.656263 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.656252    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26e05792-d6c7-4e41-92ef-0bb340d88078" containerName="main"
Jul 13 22:38:04.656309 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.656265    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="26e05792-d6c7-4e41-92ef-0bb340d88078" containerName="main"
Jul 13 22:38:04.656309 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.656277    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f756c06a-05ba-4bbc-91e5-6c5627463662" containerName="main"
Jul 13 22:38:04.656309 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.656282    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f756c06a-05ba-4bbc-91e5-6c5627463662" containerName="main"
Jul 13 22:38:04.656398 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.656335    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="26e05792-d6c7-4e41-92ef-0bb340d88078" containerName="main"
Jul 13 22:38:04.656398 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.656347    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f756c06a-05ba-4bbc-91e5-6c5627463662" containerName="main"
Jul 13 22:38:04.659262 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.659246    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7556-140665183697776"
Jul 13 22:38:04.661664 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.661641    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-gr4sb\""
Jul 13 22:38:04.666018 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.665997    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f756c06a-05ba-4bbc-91e5-6c5627463662" path="/var/lib/kubelet/pods/f756c06a-05ba-4bbc-91e5-6c5627463662/volumes"
Jul 13 22:38:04.666385 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.666366    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7556-140665183697776"]
Jul 13 22:38:04.785134 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.785111    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkwd7\" (UniqueName: \"kubernetes.io/projected/ef37401a-505b-4c44-a31d-250cae891767-kube-api-access-zkwd7\") pod \"test-curl-7556-140665183697776\" (UID: \"ef37401a-505b-4c44-a31d-250cae891767\") " pod="openshift-ingress/test-curl-7556-140665183697776"
Jul 13 22:38:04.886464 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.886436    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkwd7\" (UniqueName: \"kubernetes.io/projected/ef37401a-505b-4c44-a31d-250cae891767-kube-api-access-zkwd7\") pod \"test-curl-7556-140665183697776\" (UID: \"ef37401a-505b-4c44-a31d-250cae891767\") " pod="openshift-ingress/test-curl-7556-140665183697776"
Jul 13 22:38:04.895469 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.895441    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkwd7\" (UniqueName: \"kubernetes.io/projected/ef37401a-505b-4c44-a31d-250cae891767-kube-api-access-zkwd7\") pod \"test-curl-7556-140665183697776\" (UID: \"ef37401a-505b-4c44-a31d-250cae891767\") " pod="openshift-ingress/test-curl-7556-140665183697776"
Jul 13 22:38:04.970529 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:04.970474    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7556-140665183697776"
Jul 13 22:38:05.024455 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:05.024429    2558 generic.go:356] "Generic (PLEG): container finished" podID="26e05792-d6c7-4e41-92ef-0bb340d88078" containerID="e39361cd37691433d9c6a1d97ddb650c019c6656883c76b77f4f8583c33497dc" exitCode=0
Jul 13 22:38:05.024813 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:05.024476    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"
Jul 13 22:38:05.024813 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:05.024509    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz" event={"ID":"26e05792-d6c7-4e41-92ef-0bb340d88078","Type":"ContainerDied","Data":"e39361cd37691433d9c6a1d97ddb650c019c6656883c76b77f4f8583c33497dc"}
Jul 13 22:38:05.024813 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:05.024577    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz" event={"ID":"26e05792-d6c7-4e41-92ef-0bb340d88078","Type":"ContainerDied","Data":"f1a3a13d4120e694197a85b33c6a5611c08793162c710763badb787b4eba3c16"}
Jul 13 22:38:05.024813 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:05.024603    2558 scope.go:117] "RemoveContainer" containerID="e39361cd37691433d9c6a1d97ddb650c019c6656883c76b77f4f8583c33497dc"
Jul 13 22:38:05.035273 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:05.035256    2558 scope.go:117] "RemoveContainer" containerID="e39361cd37691433d9c6a1d97ddb650c019c6656883c76b77f4f8583c33497dc"
Jul 13 22:38:05.035597 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:38:05.035579    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e39361cd37691433d9c6a1d97ddb650c019c6656883c76b77f4f8583c33497dc\": container with ID starting with e39361cd37691433d9c6a1d97ddb650c019c6656883c76b77f4f8583c33497dc not found: ID does not exist" containerID="e39361cd37691433d9c6a1d97ddb650c019c6656883c76b77f4f8583c33497dc"
Jul 13 22:38:05.035656 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:05.035607    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e39361cd37691433d9c6a1d97ddb650c019c6656883c76b77f4f8583c33497dc"} err="failed to get container status \"e39361cd37691433d9c6a1d97ddb650c019c6656883c76b77f4f8583c33497dc\": rpc error: code = NotFound desc = could not find container \"e39361cd37691433d9c6a1d97ddb650c019c6656883c76b77f4f8583c33497dc\": container with ID starting with e39361cd37691433d9c6a1d97ddb650c019c6656883c76b77f4f8583c33497dc not found: ID does not exist"
Jul 13 22:38:05.041554 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:05.041500    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"]
Jul 13 22:38:05.045156 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:05.045136    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4b5279/rate-test-model-4b5279-kserve-5cbbddff75-lgwzz"]
Jul 13 22:38:05.099076 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:05.099049    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7556-140665183697776"]
Jul 13 22:38:05.101894 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:38:05.101866    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef37401a_505b_4c44_a31d_250cae891767.slice/crio-e4a4b5185f602582cbbc4a084b036733dc94956d1ea76981722d0611f007b70e WatchSource:0}: Error finding container e4a4b5185f602582cbbc4a084b036733dc94956d1ea76981722d0611f007b70e: Status 404 returned error can't find the container with id e4a4b5185f602582cbbc4a084b036733dc94956d1ea76981722d0611f007b70e
Jul 13 22:38:05.103115 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:05.103096    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 22:38:06.032798 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:06.032773    2558 generic.go:356] "Generic (PLEG): container finished" podID="ef37401a-505b-4c44-a31d-250cae891767" containerID="64fea83d5a72c824824971b06fec50e8ff3dccdb248425d4dc84ba2a4892c7cf" exitCode=0
Jul 13 22:38:06.033139 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:06.032861    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7556-140665183697776" event={"ID":"ef37401a-505b-4c44-a31d-250cae891767","Type":"ContainerDied","Data":"64fea83d5a72c824824971b06fec50e8ff3dccdb248425d4dc84ba2a4892c7cf"}
Jul 13 22:38:06.033139 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:06.032898    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7556-140665183697776" event={"ID":"ef37401a-505b-4c44-a31d-250cae891767","Type":"ContainerStarted","Data":"e4a4b5185f602582cbbc4a084b036733dc94956d1ea76981722d0611f007b70e"}
Jul 13 22:38:06.665673 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:06.665644    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26e05792-d6c7-4e41-92ef-0bb340d88078" path="/var/lib/kubelet/pods/26e05792-d6c7-4e41-92ef-0bb340d88078/volumes"
Jul 13 22:38:07.168712 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:07.168690    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7556-140665183697776"
Jul 13 22:38:07.201667 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:07.201645    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7556-140665183697776_ef37401a-505b-4c44-a31d-250cae891767/test-curl-7556-140665183697776/0.log"
Jul 13 22:38:07.226527 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:07.226504    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7556-140665183697776"]
Jul 13 22:38:07.231102 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:07.231084    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7556-140665183697776"]
Jul 13 22:38:07.308394 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:07.308349    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkwd7\" (UniqueName: \"kubernetes.io/projected/ef37401a-505b-4c44-a31d-250cae891767-kube-api-access-zkwd7\") pod \"ef37401a-505b-4c44-a31d-250cae891767\" (UID: \"ef37401a-505b-4c44-a31d-250cae891767\") "
Jul 13 22:38:07.310400 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:07.310377    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef37401a-505b-4c44-a31d-250cae891767-kube-api-access-zkwd7" (OuterVolumeSpecName: "kube-api-access-zkwd7") pod "ef37401a-505b-4c44-a31d-250cae891767" (UID: "ef37401a-505b-4c44-a31d-250cae891767"). InnerVolumeSpecName "kube-api-access-zkwd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:38:07.409414 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:07.409389    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkwd7\" (UniqueName: \"kubernetes.io/projected/ef37401a-505b-4c44-a31d-250cae891767-kube-api-access-zkwd7\") on node \"ip-10-0-143-101.ec2.internal\" DevicePath \"\""
Jul 13 22:38:08.043590 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:08.043560    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7556-140665183697776"
Jul 13 22:38:08.043590 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:08.043590    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4a4b5185f602582cbbc4a084b036733dc94956d1ea76981722d0611f007b70e"
Jul 13 22:38:08.053361 ip-10-0-143-101 kubenswrapper[2558]: E0713 22:38:08.053337    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7556-140665183697776\" is forbidden: User \"system:node:ip-10-0-143-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-101.ec2.internal' and this object" podUID="ef37401a-505b-4c44-a31d-250cae891767" pod="openshift-ingress/test-curl-7556-140665183697776"
Jul 13 22:38:08.665840 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:08.665806    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef37401a-505b-4c44-a31d-250cae891767" path="/var/lib/kubelet/pods/ef37401a-505b-4c44-a31d-250cae891767/volumes"
Jul 13 22:38:12.944114 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:12.944081    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:38:22.110647 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:22.110617    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65d7d75cf8-5fbt8_0956ddbd-8f58-41ee-b1df-24d976b0d677/authorino/0.log"
Jul 13 22:38:26.026621 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:26.026578    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7r2wx"]
Jul 13 22:38:27.662066 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:27.662034    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-787f66975d-v5v7q_8c7d2f0f-00fa-4176-a8ee-675c699ccf55/copy-manifests/0.log"
Jul 13 22:38:27.669151 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:27.669128    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-787f66975d-v5v7q_8c7d2f0f-00fa-4176-a8ee-675c699ccf55/manager/0.log"
Jul 13 22:38:29.540987 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:29.540949    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65d7d75cf8-5fbt8_0956ddbd-8f58-41ee-b1df-24d976b0d677/authorino/0.log"
Jul 13 22:38:29.870487 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:29.870412    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-rr499_c377887a-d696-4434-9f6c-3d3e621e5a26/kuadrant-console-plugin/0.log"
Jul 13 22:38:30.239406 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:30.239381    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-7r2wx_ebf4ce38-1577-4758-aa22-2331d2a89221/limitador/0.log"
Jul 13 22:38:30.361480 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:30.361447    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-ztb7q_52f2613e-f7a2-4c13-b472-d22231443c1c/manager/0.log"
Jul 13 22:38:31.038825 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:31.038792    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-766bd547bb-vbd5d_e6e60169-9227-432c-ad99-13551dbc761a/kube-auth-proxy/0.log"
Jul 13 22:38:32.302393 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:32.302362    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-gtf7j_b0da60a9-14dc-4fc9-8c87-7da5b0afdeab/main/0.log"
Jul 13 22:38:32.413488 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:32.413460    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-7wtw2_1b0cab3b-63be-45e0-8a02-8cb0afbb26fd/main/0.log"
Jul 13 22:38:32.753044 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:32.753012    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:38:32.757004 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:32.756985    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x6k99_68f721b5-75ba-441c-a277-cb34adc92f53/ovn-acl-logging/0.log"
Jul 13 22:38:39.766404 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:39.766371    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7n4kx_191c77b5-4225-4b5d-9402-ccdc1765dde8/global-pull-secret-syncer/0.log"
Jul 13 22:38:39.938092 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:39.938066    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-k5gj5_02ee941b-f7e8-49a2-ad16-bf6c1da015ec/konnectivity-agent/0.log"
Jul 13 22:38:40.048648 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:40.048588    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-101.ec2.internal_3b7f01a0953299e98dac4176c400f9a5/haproxy/0.log"
Jul 13 22:38:44.828610 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:44.828514    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65d7d75cf8-5fbt8_0956ddbd-8f58-41ee-b1df-24d976b0d677/authorino/0.log"
Jul 13 22:38:44.952803 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:44.952778    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-rr499_c377887a-d696-4434-9f6c-3d3e621e5a26/kuadrant-console-plugin/0.log"
Jul 13 22:38:45.156292 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:45.156263    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-7r2wx_ebf4ce38-1577-4758-aa22-2331d2a89221/limitador/0.log"
Jul 13 22:38:45.253575 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:45.253528    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-ztb7q_52f2613e-f7a2-4c13-b472-d22231443c1c/manager/0.log"
Jul 13 22:38:46.752919 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:46.752896    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4f24bf8c-c597-4c3f-87fd-6b321dd0d900/alertmanager/0.log"
Jul 13 22:38:46.784461 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:46.784433    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4f24bf8c-c597-4c3f-87fd-6b321dd0d900/config-reloader/0.log"
Jul 13 22:38:46.805695 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:46.805672    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4f24bf8c-c597-4c3f-87fd-6b321dd0d900/kube-rbac-proxy-web/0.log"
Jul 13 22:38:46.828489 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:46.828462    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4f24bf8c-c597-4c3f-87fd-6b321dd0d900/kube-rbac-proxy/0.log"
Jul 13 22:38:46.850569 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:46.850534    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4f24bf8c-c597-4c3f-87fd-6b321dd0d900/kube-rbac-proxy-metric/0.log"
Jul 13 22:38:46.868809 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:46.868790    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4f24bf8c-c597-4c3f-87fd-6b321dd0d900/prom-label-proxy/0.log"
Jul 13 22:38:46.888704 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:46.888688    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4f24bf8c-c597-4c3f-87fd-6b321dd0d900/init-config-reloader/0.log"
Jul 13 22:38:46.963892 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:46.963870    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-5xtg7_c3b45145-5be1-4091-9890-fd896bdc1b55/kube-state-metrics/0.log"
Jul 13 22:38:46.992277 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:46.992258    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-5xtg7_c3b45145-5be1-4091-9890-fd896bdc1b55/kube-rbac-proxy-main/0.log"
Jul 13 22:38:47.012997 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.012975    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-5xtg7_c3b45145-5be1-4091-9890-fd896bdc1b55/kube-rbac-proxy-self/0.log"
Jul 13 22:38:47.088053 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.088031    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-lwxhg_b663c68d-d2a4-4900-812e-a3242a12eb93/monitoring-plugin/0.log"
Jul 13 22:38:47.132267 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.132238    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hvp9r_306c0019-4dd3-4c1e-885b-77dba7eb9c26/node-exporter/0.log"
Jul 13 22:38:47.154411 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.154387    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hvp9r_306c0019-4dd3-4c1e-885b-77dba7eb9c26/kube-rbac-proxy/0.log"
Jul 13 22:38:47.177395 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.177376    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hvp9r_306c0019-4dd3-4c1e-885b-77dba7eb9c26/init-textfile/0.log"
Jul 13 22:38:47.379827 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.379757    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wjh7h_1fd51438-aa61-4d28-807a-6e48e4f2debb/kube-rbac-proxy-main/0.log"
Jul 13 22:38:47.399076 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.399052    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wjh7h_1fd51438-aa61-4d28-807a-6e48e4f2debb/kube-rbac-proxy-self/0.log"
Jul 13 22:38:47.419099 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.419076    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-wjh7h_1fd51438-aa61-4d28-807a-6e48e4f2debb/openshift-state-metrics/0.log"
Jul 13 22:38:47.454697 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.454675    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5530f91e-4527-467a-94c4-7c14bd6efe21/prometheus/0.log"
Jul 13 22:38:47.475325 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.475280    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5530f91e-4527-467a-94c4-7c14bd6efe21/config-reloader/0.log"
Jul 13 22:38:47.495679 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.495663    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5530f91e-4527-467a-94c4-7c14bd6efe21/thanos-sidecar/0.log"
Jul 13 22:38:47.517105 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.517085    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5530f91e-4527-467a-94c4-7c14bd6efe21/kube-rbac-proxy-web/0.log"
Jul 13 22:38:47.538264 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.538244    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5530f91e-4527-467a-94c4-7c14bd6efe21/kube-rbac-proxy/0.log"
Jul 13 22:38:47.556859 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.556843    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5530f91e-4527-467a-94c4-7c14bd6efe21/kube-rbac-proxy-thanos/0.log"
Jul 13 22:38:47.577595 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.577577    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5530f91e-4527-467a-94c4-7c14bd6efe21/init-config-reloader/0.log"
Jul 13 22:38:47.609002 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.608979    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-c22nr_22d05f1c-bcb1-4a97-92cb-078a1025393d/prometheus-operator/0.log"
Jul 13 22:38:47.626098 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:47.626079    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-c22nr_22d05f1c-bcb1-4a97-92cb-078a1025393d/kube-rbac-proxy/0.log"
Jul 13 22:38:49.582817 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.582784    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"]
Jul 13 22:38:49.583181 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.583150    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef37401a-505b-4c44-a31d-250cae891767" containerName="test-curl-7556-140665183697776"
Jul 13 22:38:49.583181 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.583161    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef37401a-505b-4c44-a31d-250cae891767" containerName="test-curl-7556-140665183697776"
Jul 13 22:38:49.583254 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.583229    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef37401a-505b-4c44-a31d-250cae891767" containerName="test-curl-7556-140665183697776"
Jul 13 22:38:49.588306 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.588287    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.591046 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.591028    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5pw76\"/\"openshift-service-ca.crt\""
Jul 13 22:38:49.592061 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.592031    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5pw76\"/\"default-dockercfg-8z5qb\""
Jul 13 22:38:49.592061 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.592054    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5pw76\"/\"kube-root-ca.crt\""
Jul 13 22:38:49.596867 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.596843    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"]
Jul 13 22:38:49.743957 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.743935    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/125ff7a3-db07-4e3e-a230-592ee882188e-podres\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.744087 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.743994    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/125ff7a3-db07-4e3e-a230-592ee882188e-proc\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.744087 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.744051    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54nl9\" (UniqueName: \"kubernetes.io/projected/125ff7a3-db07-4e3e-a230-592ee882188e-kube-api-access-54nl9\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.744189 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.744132    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/125ff7a3-db07-4e3e-a230-592ee882188e-sys\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.744298 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.744281    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/125ff7a3-db07-4e3e-a230-592ee882188e-lib-modules\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.845373 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.845316    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/125ff7a3-db07-4e3e-a230-592ee882188e-podres\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.845373 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.845357    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/125ff7a3-db07-4e3e-a230-592ee882188e-proc\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.845527 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.845378    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54nl9\" (UniqueName: \"kubernetes.io/projected/125ff7a3-db07-4e3e-a230-592ee882188e-kube-api-access-54nl9\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.845527 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.845447    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/125ff7a3-db07-4e3e-a230-592ee882188e-proc\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.845527 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.845500    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/125ff7a3-db07-4e3e-a230-592ee882188e-podres\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.845661 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.845557    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/125ff7a3-db07-4e3e-a230-592ee882188e-sys\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.845661 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.845575    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/125ff7a3-db07-4e3e-a230-592ee882188e-sys\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.845661 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.845615    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/125ff7a3-db07-4e3e-a230-592ee882188e-lib-modules\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.845793 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.845707    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/125ff7a3-db07-4e3e-a230-592ee882188e-lib-modules\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.853119 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.853101    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54nl9\" (UniqueName: \"kubernetes.io/projected/125ff7a3-db07-4e3e-a230-592ee882188e-kube-api-access-54nl9\") pod \"perf-node-gather-daemonset-bv7q7\" (UID: \"125ff7a3-db07-4e3e-a230-592ee882188e\") " pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:49.899428 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:49.899409    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:50.030470 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:50.030448    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"]
Jul 13 22:38:50.032848 ip-10-0-143-101 kubenswrapper[2558]: W0713 22:38:50.032820    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod125ff7a3_db07_4e3e_a230_592ee882188e.slice/crio-7476731fa17e6948f00bf602d604d8048d5d3a880ebb9ca78ea780ed3140567b WatchSource:0}: Error finding container 7476731fa17e6948f00bf602d604d8048d5d3a880ebb9ca78ea780ed3140567b: Status 404 returned error can't find the container with id 7476731fa17e6948f00bf602d604d8048d5d3a880ebb9ca78ea780ed3140567b
Jul 13 22:38:50.232766 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:50.232737    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7" event={"ID":"125ff7a3-db07-4e3e-a230-592ee882188e","Type":"ContainerStarted","Data":"1709e8dca7f7f708c48be65305f43cfe74710f9709cdfe50d2ac23dd07d007da"}
Jul 13 22:38:50.232866 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:50.232786    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7" event={"ID":"125ff7a3-db07-4e3e-a230-592ee882188e","Type":"ContainerStarted","Data":"7476731fa17e6948f00bf602d604d8048d5d3a880ebb9ca78ea780ed3140567b"}
Jul 13 22:38:50.232919 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:50.232880    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:50.248781 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:50.248719    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7" podStartSLOduration=1.248708033 podStartE2EDuration="1.248708033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 22:38:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:38:50.247944335 +0000 UTC m=+2418.165920798" watchObservedRunningTime="2026-07-13 22:38:50.248708033 +0000 UTC m=+2418.166684496"
Jul 13 22:38:52.064258 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:52.064227    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b57f6_e82d0f20-1cb3-4f5f-a04e-3e10b5f76593/dns/0.log"
Jul 13 22:38:52.081986 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:52.081957    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b57f6_e82d0f20-1cb3-4f5f-a04e-3e10b5f76593/kube-rbac-proxy/0.log"
Jul 13 22:38:52.170222 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:52.170192    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p75ql_1820ff41-2d99-4666-b910-90b3110b7e32/dns-node-resolver/0.log"
Jul 13 22:38:52.786938 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:52.786915    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w98nt_a27ef85e-b4e2-4778-855e-7a05bfeb4a00/node-ca/0.log"
Jul 13 22:38:53.906417 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:53.906387    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-766bd547bb-vbd5d_e6e60169-9227-432c-ad99-13551dbc761a/kube-auth-proxy/0.log"
Jul 13 22:38:54.798275 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:54.798227    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wjjwx_a7d9e375-8217-427d-81d9-ae0aaa546025/serve-healthcheck-canary/0.log"
Jul 13 22:38:55.388681 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:55.388652    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jsl8d_dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b/kube-rbac-proxy/0.log"
Jul 13 22:38:55.408963 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:55.408925    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jsl8d_dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b/exporter/0.log"
Jul 13 22:38:55.428523 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:55.428499    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jsl8d_dc2f9a6c-cdf8-48f5-a66e-507c0dcbfe2b/extractor/0.log"
Jul 13 22:38:56.247038 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:56.247012    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5pw76/perf-node-gather-daemonset-bv7q7"
Jul 13 22:38:57.979079 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:57.979052    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-787f66975d-v5v7q_8c7d2f0f-00fa-4176-a8ee-675c699ccf55/manager/0.log"
Jul 13 22:38:57.998750 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:57.998728    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-787f66975d-v5v7q_8c7d2f0f-00fa-4176-a8ee-675c699ccf55/copy-manifests/0.log"
Jul 13 22:38:59.492482 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:38:59.492447    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-849844fc89-qzcbm_2a3bd4ed-f867-4159-abfd-ff23ee03d0e1/manager/0.log"
Jul 13 22:39:00.719808 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:39:00.719768    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-766bd547bb-vbd5d_e6e60169-9227-432c-ad99-13551dbc761a/kube-auth-proxy/0.log"
Jul 13 22:39:02.222463 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:39:02.222435    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-766bd547bb-vbd5d_e6e60169-9227-432c-ad99-13551dbc761a/kube-auth-proxy/0.log"
Jul 13 22:39:06.069077 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:39:06.069049    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-czcbx_2948a7e0-5979-4656-9e00-c2cdde279ab2/kube-multus-additional-cni-plugins/0.log"
Jul 13 22:39:06.088072 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:39:06.088049    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-czcbx_2948a7e0-5979-4656-9e00-c2cdde279ab2/egress-router-binary-copy/0.log"
Jul 13 22:39:06.106294 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:39:06.106262    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-czcbx_2948a7e0-5979-4656-9e00-c2cdde279ab2/cni-plugins/0.log"
Jul 13 22:39:06.123750 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:39:06.123727    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-czcbx_2948a7e0-5979-4656-9e00-c2cdde279ab2/bond-cni-plugin/0.log"
Jul 13 22:39:06.141273 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:39:06.141249    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-czcbx_2948a7e0-5979-4656-9e00-c2cdde279ab2/routeoverride-cni/0.log"
Jul 13 22:39:06.159149 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:39:06.159129    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-czcbx_2948a7e0-5979-4656-9e00-c2cdde279ab2/whereabouts-cni-bincopy/0.log"
Jul 13 22:39:06.176434 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:39:06.176415    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-czcbx_2948a7e0-5979-4656-9e00-c2cdde279ab2/whereabouts-cni/0.log"
Jul 13 22:39:06.560989 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:39:06.560963    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vht6c_805523c8-888a-45cb-8291-6ee665e8eeb1/kube-multus/0.log"
Jul 13 22:39:06.604712 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:39:06.604691    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bx7hm_b485735e-e460-453b-8503-c034e753bf78/network-metrics-daemon/0.log"
Jul 13 22:39:06.620025 ip-10-0-143-101 kubenswrapper[2558]: I0713 22:39:06.619999    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bx7hm_b485735e-e460-453b-8503-c034e753bf78/kube-rbac-proxy/0.log"