Jul 03 09:41:16.984188 ip-10-0-138-129 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 03 09:41:16.984201 ip-10-0-138-129 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 03 09:41:16.984210 ip-10-0-138-129 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 03 09:41:16.984540 ip-10-0-138-129 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 03 09:41:27.060364 ip-10-0-138-129 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Jul 03 09:41:27.060378 ip-10-0-138-129 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7d95fadf0c3c4c5594b6665569bb1b9f --
Jul 03 09:43:18.016147 ip-10-0-138-129 systemd[1]: Starting Kubernetes Kubelet...
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.475615    2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482704    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482719    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482723    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482728    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482731    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482733    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482736    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 09:43:18.561139 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482739    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482742    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482744    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482747    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482750    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482775    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482779    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482783    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482786    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482789    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482793    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482797    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482800    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482803    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482806    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482809    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482811    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482814    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482816    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 09:43:18.562172 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482819    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482822    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482824    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482827    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482844    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482848    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482852    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482855    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482858    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482861    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482864    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482867    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482870    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482873    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482876    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482879    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482882    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482884    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482887    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482889    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 09:43:18.564102 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482892    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 09:43:18.563154 ip-10-0-138-129 systemd[1]: Started Kubernetes Kubelet.
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482895    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482899    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482902    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482905    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482908    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482910    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482913    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482915    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482918    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482921    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482923    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482926    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482928    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482932    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482936    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482939    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482941    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482945    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482947    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 09:43:18.565224 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482950    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482952    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482955    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482957    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482960    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482963    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482966    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482968    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482971    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482974    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482976    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482979    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482982    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482985    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482987    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482991    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482994    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.482997    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483000    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 09:43:18.566482 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483002    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483005    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483007    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483010    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483012    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483016    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483019    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483022    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483024    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483027    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483030    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483034    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483038    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483041    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483755    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483761    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483764    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483767    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483770    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 09:43:18.567116 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483773    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483776    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483779    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483782    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483785    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483788    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483791    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483795    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483798    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483801    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483803    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483806    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483809    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483812    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483815    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483817    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483820    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483823    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483825    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483828    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 09:43:18.567882 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483846    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483850    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483853    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483855    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483858    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483861    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483865    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483867    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483870    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483873    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483876    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483878    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483882    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483885    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483888    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483890    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483893    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483897    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483901    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 09:43:18.568530 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483904    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483907    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483910    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483913    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483916    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483918    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483921    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483923    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483925    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483929    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483931    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483934    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483936    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483939    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483942    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483944    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483947    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483949    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483952    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 09:43:18.569220 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483955    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483958    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483963    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483966    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483970    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483972    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483975    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483978    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483980    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483983    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483986    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483988    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483991    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483993    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483996    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.483998    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484001    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484004    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484006    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484009    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 09:43:18.569782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484012    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484014    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484017    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484019    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484022    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484024    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484027    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484029    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484034    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484037    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484040    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484043    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484045    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484048    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484052    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484055    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484431    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484435    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484438    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484441    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 09:43:18.570613 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484444    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484446    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484449    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484452    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484455    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484457    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484460    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484463    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484466    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484469    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484471    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484474    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484477    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484479    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484482    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484485    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484487    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484490    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484493    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 09:43:18.571245 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484495    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484498    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484500    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484503    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484505    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484508    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484511    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484514    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484516    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484519    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484521    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484524    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484527    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484529    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484532    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484535    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484537    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484540    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484542    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 09:43:18.691853 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484547    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484554    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484558    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484561    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484563    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484567    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484569    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484572    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484575    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484578    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484581    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484584    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484587    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484589    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484592    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484594    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484597    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484599    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484602    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484604    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 09:43:18.692592 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484607    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484610    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484613    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484616    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484618    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484621    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484624    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484626    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484630    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484634    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484637    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484640    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484644    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484649    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484653    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484656    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484659    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484662    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484665    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 09:43:18.693399 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484667    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484670    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484672    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484675    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484677    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484680    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484683    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484685    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484688    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484690    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484693    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484695    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484698    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484701    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484703    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484706    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484709    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.484711    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484780    2571 flags.go:64] FLAG: --address="0.0.0.0"
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484787    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484793    2571 flags.go:64] FLAG: --anonymous-auth="true"
Jul 03 09:43:18.694325 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484798    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484802    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484824    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484842    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484848    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484853    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484856    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484860    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484863    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484866    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484869    2571 flags.go:64] FLAG: --cgroup-root=""
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484872    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484875    2571 flags.go:64] FLAG: --client-ca-file=""
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484878    2571 flags.go:64] FLAG: --cloud-provider="external"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484881    2571 flags.go:64] FLAG: --cluster-dns="[]"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484887    2571 flags.go:64] FLAG: --cluster-domain=""
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484890    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484893    2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484896    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484899    2571 flags.go:64] FLAG: --container-log-max-files="5"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484903    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484907    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484910    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 03 09:43:18.694978 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484913    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484916    2571 flags.go:64] FLAG: --contention-profiling="false"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484919    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484922    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484926    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484929    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484933    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484936    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484938    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484942    2571 flags.go:64] FLAG: --enable-load-reader="false"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484945    2571 flags.go:64] FLAG: --enable-server="true"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484948    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484952    2571 flags.go:64] FLAG: --event-burst="100"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484955    2571 flags.go:64] FLAG: --event-qps="50"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484960    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484963    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484966    2571 flags.go:64] FLAG: --eviction-hard=""
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484970    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484973    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484976    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484979    2571 flags.go:64] FLAG: --eviction-soft=""
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484982    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484985    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484988    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484991    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 03 09:43:18.695641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484994    2571 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.484997    2571 flags.go:64] FLAG: --fail-swap-on="true"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485000    2571 flags.go:64] FLAG: --feature-gates=""
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485003    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485007    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485010    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485013    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485016    2571 flags.go:64] FLAG: --healthz-port="10248"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485019    2571 flags.go:64] FLAG: --help="false"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485022    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485025    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485028    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485031    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485035    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485039    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485041    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485044    2571 flags.go:64] FLAG: --image-service-endpoint=""
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485047    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485050    2571 flags.go:64] FLAG: --kube-api-burst="100"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485053    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485057    2571 flags.go:64] FLAG: --kube-api-qps="50"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485061    2571 flags.go:64] FLAG: --kube-reserved=""
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485064    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485069    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 03 09:43:18.698081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485072    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485075    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485077    2571 flags.go:64] FLAG: --lock-file=""
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485080    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485083    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485087    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485093    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485095    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485098    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485101    2571 flags.go:64] FLAG: --logging-format="text"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485104    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485107    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485110    2571 flags.go:64] FLAG: --manifest-url=""
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485113    2571 flags.go:64] FLAG: --manifest-url-header=""
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485118    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485121    2571 flags.go:64] FLAG: --max-open-files="1000000"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485125    2571 flags.go:64] FLAG: --max-pods="110"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485128    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485131    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485134    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485137    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485140    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485143    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485146    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485154    2571 flags.go:64] FLAG: --node-status-max-images="50"
Jul 03 09:43:18.699003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485157    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485160    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485163    2571 flags.go:64] FLAG: --pod-cidr=""
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485166    2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485173    2571 flags.go:64] FLAG: --pod-manifest-path=""
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485176    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485179    2571 flags.go:64] FLAG: --pods-per-core="0"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485185    2571 flags.go:64] FLAG: --port="10250"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485188    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485191    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f1ad4f53106ae9a3"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485194    2571 flags.go:64] FLAG: --qos-reserved=""
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485197    2571 flags.go:64] FLAG: --read-only-port="10255"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485200    2571 flags.go:64] FLAG: --register-node="true"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485203    2571 flags.go:64] FLAG: --register-with-taints=""
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485207    2571 flags.go:64] FLAG: --registry-burst="10"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485210    2571 flags.go:64] FLAG: --registry-qps="5"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485212    2571 flags.go:64] FLAG: --reserved-cpus=""
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485215    2571 flags.go:64] FLAG: --reserved-memory=""
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485219    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485222    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485225    2571 flags.go:64] FLAG: --rotate-certificates="false"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485228    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485231    2571 flags.go:64] FLAG: --runonce="false"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485234    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485237    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 03 09:43:18.712485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485240    2571 flags.go:64] FLAG: --seccomp-default="false"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485242    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485245    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485248    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485252    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485255    2571 flags.go:64] FLAG: --storage-driver-password="root"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485258    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485261    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485264    2571 flags.go:64] FLAG: --storage-driver-user="root"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485267    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485270    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485274    2571 flags.go:64] FLAG: --system-cgroups=""
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485277    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485282    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485286    2571 flags.go:64] FLAG: --tls-cert-file=""
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485289    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485293    2571 flags.go:64] FLAG: --tls-min-version=""
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485296    2571 flags.go:64] FLAG: --tls-private-key-file=""
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485299    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485302    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485305    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485308    2571 flags.go:64] FLAG: --v="2"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485312    2571 flags.go:64] FLAG: --version="false"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485317    2571 flags.go:64] FLAG: --vmodule=""
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485320    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 03 09:43:18.713335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485324    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485443    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485448    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485452    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485456    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485459    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485462    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485465    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485468    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485471    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485473    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485476    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485481    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485484    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485487    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485489    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485492    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485495    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485499    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 09:43:18.807326 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485502    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485505    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485509    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485512    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485515    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485518    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485520    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485523    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485526    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485528    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485531    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485534    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485536    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485539    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485541    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485544    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485546    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485549    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485551    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485554    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 09:43:18.808407 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485556    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485559    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485561    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485564    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485566    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485569    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485572    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485574    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485577    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485579    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485582    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485586    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485588    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485591    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485594    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485597    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485600    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485603    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485605    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485608    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 09:43:18.814723 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485610    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485613    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485616    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485618    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485621    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485624    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485627    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485629    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485632    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485634    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485637    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485639    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485641    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485644    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485647    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485649    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485652    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485654    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485657    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485660    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 09:43:18.815740 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485663    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485665    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485668    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485671    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485674    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485676    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485680    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485682    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485685    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485688    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485690    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485693    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485695    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485698    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485700    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485703    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485721    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485724    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485727    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485730    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 09:43:18.816531 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.485732    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.485738    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.492742    2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.492760    2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.493459    2571 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.493466    2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493527    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493533    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493536    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493539    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493542    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493545    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493548    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493551    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493554    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 09:43:18.880142 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493557    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493560    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493562    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493565    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493568    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493576    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493579    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493582    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493585    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493588    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493591    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493594    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493596    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493599    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493602    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493605    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493608    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493610    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493613    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 09:43:18.880889 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493616    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493619    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493622    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493625    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493627    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493630    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493632    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493635    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493637    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493640    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493643    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493645    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493648    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493665    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493669    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493672    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493675    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493678    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493682    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 09:43:18.881880 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493685    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493688    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493692    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493696    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493699    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493703    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493714    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493718    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493722    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493726    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493730    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493733    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493736    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493739    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493742    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493746    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493748    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493751    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493754    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 09:43:18.882635 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493756    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493759    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493762    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493764    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493767    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493769    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493772    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493775    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493778    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493780    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493783    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493785    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493789    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493792    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493795    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493798    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493801    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493804    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493806    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493809    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 09:43:18.883638 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493812    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493814    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493817    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493820    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493822    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493825    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493844    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493848    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493851    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493854    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493857    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493860    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493862    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.493867    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493990    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 09:43:18.885170 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493995    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.493998    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494001    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494005    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494010    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494013    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494015    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494018    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494021    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494024    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494028    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494031    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494033    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494036    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494039    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494041    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494044    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494046    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494049    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 09:43:18.885640 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494052    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494054    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494057    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494060    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494066    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494069    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494072    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494074    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494077    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494081    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494085    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494088    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494091    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494093    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494096    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494100    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494102    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494105    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494107    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494110    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 09:43:18.886177 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494112    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494115    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494117    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494121    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494124    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494127    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494129    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494132    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494134    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494137    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494140    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494142    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494146    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494149    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494152    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494155    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494160    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494163    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494166    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494169    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 09:43:18.886729 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494172    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494175    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494178    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494181    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494184    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494187    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494190    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494193    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494195    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494198    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494200    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494203    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494205    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494208    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494211    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494215    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494218    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494220    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494223    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 09:43:18.887284 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494226    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494228    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494231    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494234    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494236    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494239    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494241    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494244    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494247    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494250    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494253    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494256    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494258    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494261    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494264    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494267    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494270    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494272    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494275    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 09:43:18.887801 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:18.494277    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 09:43:18.888343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.494282    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 03 09:43:18.888343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.494406    2571 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 03 09:43:18.888343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.496370    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 03 09:43:18.888343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.497725    2571 server.go:1019] "Starting client certificate rotation"
Jul 03 09:43:18.888343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.497846    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 03 09:43:18.888343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.497905    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 03 09:43:18.888343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.504846    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 03 09:43:18.888343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.506035    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 03 09:43:18.888343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.522498    2571 log.go:25] "Validated CRI v1 runtime API"
Jul 03 09:43:18.888343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.524635    2571 log.go:25] "Validated CRI v1 image API"
Jul 03 09:43:18.888343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.525469    2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 03 09:43:18.888343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.530544    2571 fs.go:135] Filesystem UUIDs: map[45240d6d-43fd-484f-9b42-4f2db17c8c12:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 82bc09ab-5199-4f93-9ef0-994c07a139e2:/dev/nvme0n1p3]
Jul 03 09:43:18.888691 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.530559    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 03 09:43:18.888691 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.534360    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.535791    2571 manager.go:216] Machine: {Timestamp:2026-07-03 09:43:18.53455238 +0000 UTC m=+0.399841026 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3078565 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec211088e888beffb2e26de805d0ba13 SystemUUID:ec211088-e888-beff-b2e2-6de805d0ba13 BootID:7d95fadf-0c3c-4c55-94b6-665569bb1b9f Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e4:17:ad:19:81 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e4:17:ad:19:81 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:92:ce:9b:c5:aa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.535912    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.536044    2571 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.538926    2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.538957    2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-129.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.540060    2571 topology_manager.go:138] "Creating topology manager with none policy"
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.540072    2571 container_manager_linux.go:309] "Creating device plugin manager"
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.540079    2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.540095    2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.541531    2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.545109    2571 state_mem.go:36] "Initialized new in-memory state store"
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.545229    2571 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.545255    2571 kubelet.go:486] "Attempting to sync node with API server"
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.545266    2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.545281    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.545291    2571 kubelet.go:398] "Adding apiserver pod source"
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.545300    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 03 09:43:18.888765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.545626    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.545641    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.552733    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hxzs8"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.553030    2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.555619    2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.555646    2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.556690    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.556707    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.556716    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.556723    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.556735    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.556740    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.556748    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.556753    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.556760    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.556765    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.556773    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.556778    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.558055    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.558063    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.560981    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hxzs8"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.561954    2571 server.go:1268] "Started kubelet"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.563941    2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.564266    2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-129.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.564262    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-129.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.564332    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.564786    2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.564881    2571 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.566439    2571 server.go:310] "Adding debug handlers to kubelet server"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.566692    2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.570139    2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.571960    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.571969    2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.572049    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-129.ec2.internal\" not found"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.572713    2571 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.572719    2571 factory.go:55] Registering systemd factory
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.572729    2571 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.572736    2571 factory.go:223] Registration of the systemd container factory successfully
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.572714    2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.572989    2571 factory.go:153] Registering CRI-O factory
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.573000    2571 factory.go:223] Registration of the crio container factory successfully
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.573047    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.573068    2571 factory.go:103] Registering Raw factory
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.573084    2571 manager.go:1196] Started watching for new ooms in manager
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.573291    2571 reconstruct.go:97] "Volume reconstruction finished"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.573302    2571 reconciler.go:29] "Reconciler: start to sync state"
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.573427    2571 manager.go:349] Starting recovery of all containers
Jul 03 09:43:18.889312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.579679    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.582464    2571 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-129.ec2.internal\" not found" node="ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.589385    2571 manager.go:354] Recovery completed
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.594327    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.596329    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.596364    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.596377    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientPID"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.601090    2571 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.601101    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.601100    2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.601166    2571 state_mem.go:36] "Initialized new in-memory state store"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.601905    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.601938    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.601949    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientPID"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.602422    2571 policy_none.go:49] "None policy: Start"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.602436    2571 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.602446    2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.603364    2571 policy_none.go:47] "Start"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.649558    2571 manager.go:337] "Starting Device Plugin manager"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.649605    2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.649616    2571 server.go:87] "Starting device plugin registration server"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.649941    2571 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.649952    2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.650099    2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.650190    2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.650198    2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.650809    2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.650890    2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-129.ec2.internal\" not found"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.750265    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.751512    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.751544    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.751555    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientPID"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.751583    2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.756561    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.757765    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.757798    2571 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.757826    2571 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.757871    2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.760427    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.760756    2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.760775    2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-129.ec2.internal\": node \"ip-10-0-138-129.ec2.internal\" not found"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.858919    2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-129.ec2.internal"]
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.859027    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.860938    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.860968    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.860979    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientPID"
Jul 03 09:43:18.890636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.862191    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.862300    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.862332    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.863079    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.863099    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.863109    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientPID"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.863137    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.863157    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.863167    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientPID"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.864333    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.864364    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.865170    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.865206    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.865224    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeHasSufficientPID"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.875650    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d6f9c3a7f49c697f26f22966d3293ba5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal\" (UID: \"d6f9c3a7f49c697f26f22966d3293ba5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.875805    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6f9c3a7f49c697f26f22966d3293ba5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal\" (UID: \"d6f9c3a7f49c697f26f22966d3293ba5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.891998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.876050    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b662888df2876571019936bda200019b-config\") pod \"kube-apiserver-proxy-ip-10-0-138-129.ec2.internal\" (UID: \"b662888df2876571019936bda200019b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.899045 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.897641    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-129.ec2.internal\" not found"
Jul 03 09:43:18.907751 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.907729    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-129.ec2.internal\" not found" node="ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.912297 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.912281    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-129.ec2.internal\" not found" node="ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.976170 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.976139    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d6f9c3a7f49c697f26f22966d3293ba5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal\" (UID: \"d6f9c3a7f49c697f26f22966d3293ba5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.976170 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.976173    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6f9c3a7f49c697f26f22966d3293ba5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal\" (UID: \"d6f9c3a7f49c697f26f22966d3293ba5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.976347 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.976194    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b662888df2876571019936bda200019b-config\") pod \"kube-apiserver-proxy-ip-10-0-138-129.ec2.internal\" (UID: \"b662888df2876571019936bda200019b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.976347 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.976233    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b662888df2876571019936bda200019b-config\") pod \"kube-apiserver-proxy-ip-10-0-138-129.ec2.internal\" (UID: \"b662888df2876571019936bda200019b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.976347 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.976259    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d6f9c3a7f49c697f26f22966d3293ba5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal\" (UID: \"d6f9c3a7f49c697f26f22966d3293ba5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.976347 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:18.976277    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6f9c3a7f49c697f26f22966d3293ba5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal\" (UID: \"d6f9c3a7f49c697f26f22966d3293ba5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:18.997728 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:18.997699    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-129.ec2.internal\" not found"
Jul 03 09:43:19.099344 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.099265    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-129.ec2.internal\" not found"
Jul 03 09:43:19.200297 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.200252    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-129.ec2.internal\" not found"
Jul 03 09:43:19.211218 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.211189    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:19.216058 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.215440    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:19.301794 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.301748    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-129.ec2.internal\" not found"
Jul 03 09:43:19.403026 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.402941    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-129.ec2.internal\" not found"
Jul 03 09:43:19.498009 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.497975    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 03 09:43:19.498173 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.498116    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 03 09:43:19.498173 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.498151    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 03 09:43:19.503131 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.503103    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-129.ec2.internal\" not found"
Jul 03 09:43:19.513467 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.513439    2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 09:43:19.549843 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.549807    2571 apiserver.go:52] "Watching apiserver"
Jul 03 09:43:19.558994 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.558964    2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 03 09:43:19.559350 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.559328    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-8hmsd","kube-system/konnectivity-agent-g8hbm","openshift-dns/node-resolver-gmjss","openshift-multus/multus-4pl6g","openshift-multus/multus-additional-cni-plugins-dtsgm","openshift-multus/network-metrics-daemon-txhn2","kube-system/global-pull-secret-syncer-n4gfv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg","openshift-cluster-node-tuning-operator/tuned-r89ht","openshift-image-registry/node-ca-fng7l","openshift-network-diagnostics/network-check-target-d6fcm","openshift-network-operator/iptables-alerter-z7ljw"]
Jul 03 09:43:19.560955 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.560934    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.562110 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.562084    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g8hbm"
Jul 03 09:43:19.563147 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.563128    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gmjss"
Jul 03 09:43:19.563697 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.563680    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 03 09:43:19.563796 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.563762    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 03 09:43:19.563889 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.563764    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 03 09:43:19.563889 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.563827    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 03 09:43:19.564234 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.564215    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 03 09:43:19.564234 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.564228    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-v7wm5\""
Jul 03 09:43:19.564406 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.564278    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.564406 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.564284    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 03 09:43:19.564563 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.564519    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-02 09:38:19 +0000 UTC" deadline="2027-12-21 09:43:04.424350051 +0000 UTC"
Jul 03 09:43:19.564629 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.564563    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12863h59m44.859791183s"
Jul 03 09:43:19.564684 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.564622    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 03 09:43:19.564684 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.564640    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 03 09:43:19.564684 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.564680    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8pvr2\""
Jul 03 09:43:19.565319 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.565305    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4cvf7\""
Jul 03 09:43:19.565655 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.565638    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 03 09:43:19.565655 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.565654    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 03 09:43:19.567210 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.566368    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.567210 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.566649    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 03 09:43:19.567210 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.567124    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pxtss\""
Jul 03 09:43:19.567210 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.567153    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 03 09:43:19.567635 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.567614    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 03 09:43:19.567721 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.567699    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 03 09:43:19.568778 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.568669    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:19.568902 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.568816    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:19.568997 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.568975    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 03 09:43:19.569798 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.569778    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hq9mx\""
Jul 03 09:43:19.569912 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.569812    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 03 09:43:19.570272 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.570257    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:19.570379 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.570358    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:19.571340 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.571324    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.572071 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.572055    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 03 09:43:19.572200 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.572184    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:19.572883 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.572604    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.573515 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.573491    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 03 09:43:19.573608 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.573543    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 03 09:43:19.573608 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.573582    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 03 09:43:19.573750 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.573635    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-f2fqb\""
Jul 03 09:43:19.574001 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.573988    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fng7l"
Jul 03 09:43:19.574921 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.574899    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 03 09:43:19.575032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.575015    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8jlqr\""
Jul 03 09:43:19.575096 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.575026    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 03 09:43:19.575285 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.575271    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:19.575356 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.575340    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:19.576162 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.576149    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 03 09:43:19.576328 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.576311    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 03 09:43:19.576433 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.576416    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8fzsq\""
Jul 03 09:43:19.576537 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.576520    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z7ljw"
Jul 03 09:43:19.576601 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.576552    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 03 09:43:19.578534 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578513    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.578617 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578538    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-modprobe-d\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.578617 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578555    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-sysctl-conf\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.578617 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578577    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-run-ovn\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.578617 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578599    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/20bc336f-ff0c-4072-8275-46857fb9f016-hosts-file\") pod \"node-resolver-gmjss\" (UID: \"20bc336f-ff0c-4072-8275-46857fb9f016\") " pod="openshift-dns/node-resolver-gmjss"
Jul 03 09:43:19.578776 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578632    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.578776 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578670    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-systemd-units\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.578776 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578702    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-run-openvswitch\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.578776 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578705    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 03 09:43:19.578776 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578738    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-os-release\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.578776 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578759    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/420a7e0c-545b-437b-bf1c-6a5e24143c06-multus-daemon-config\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.579062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578783    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-run-multus-certs\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.579062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578818    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d642348a-9842-43db-bcdf-5d84ef42e828-dbus\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:19.579062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578832    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/24b56ac3-1415-4f69-aedc-44c6c2efd7dc-agent-certs\") pod \"konnectivity-agent-g8hbm\" (UID: \"24b56ac3-1415-4f69-aedc-44c6c2efd7dc\") " pod="kube-system/konnectivity-agent-g8hbm"
Jul 03 09:43:19.579062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578847    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ws686\""
Jul 03 09:43:19.579062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578871    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-sysctl-d\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.579062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578890    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa4df66b-74e4-4cde-96dc-9ae773748dd8-tmp\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.579062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578927    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 03 09:43:19.579062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578951    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 03 09:43:19.579062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578954    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-multus-socket-dir-parent\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.579062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.578987    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-device-dir\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.579062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579024    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-run\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.579062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579053    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlwt4\" (UniqueName: \"kubernetes.io/projected/aa4df66b-74e4-4cde-96dc-9ae773748dd8-kube-api-access-dlwt4\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579073    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579090    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-var-lib-cni-multus\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579120    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579141    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579157    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-sysconfig\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579178    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-run-netns\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579208    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-node-log\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579233    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d915f962-809c-4394-aa7b-72b49fdde9b2-ovn-node-metrics-cert\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579262    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579280    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/420a7e0c-545b-437b-bf1c-6a5e24143c06-cni-binary-copy\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579296    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-hostroot\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579315    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-sys-fs\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579339    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-tuned\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579368    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-kubelet\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579394    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-cni-netd\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.579483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579446    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-run-k8s-cni-cncf-io\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579480    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-slash\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579503    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d642348a-9842-43db-bcdf-5d84ef42e828-kubelet-config\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579520    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd3cac63-b412-453b-90bc-5b7897a9920f-host\") pod \"node-ca-fng7l\" (UID: \"dd3cac63-b412-453b-90bc-5b7897a9920f\") " pod="openshift-image-registry/node-ca-fng7l"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579540    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-system-cni-dir\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579557    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579591    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78fs7\" (UniqueName: \"kubernetes.io/projected/6328c973-29b4-40b7-8467-7a1439e175bc-kube-api-access-78fs7\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579606    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d915f962-809c-4394-aa7b-72b49fdde9b2-env-overrides\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579640    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkm6g\" (UniqueName: \"kubernetes.io/projected/dd3cac63-b412-453b-90bc-5b7897a9920f-kube-api-access-lkm6g\") pod \"node-ca-fng7l\" (UID: \"dd3cac63-b412-453b-90bc-5b7897a9920f\") " pod="openshift-image-registry/node-ca-fng7l"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579667    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wzc7\" (UniqueName: \"kubernetes.io/projected/20bc336f-ff0c-4072-8275-46857fb9f016-kube-api-access-6wzc7\") pod \"node-resolver-gmjss\" (UID: \"20bc336f-ff0c-4072-8275-46857fb9f016\") " pod="openshift-dns/node-resolver-gmjss"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579683    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-var-lib-kubelet\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579723    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdk7m\" (UniqueName: \"kubernetes.io/projected/420a7e0c-545b-437b-bf1c-6a5e24143c06-kube-api-access-zdk7m\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579744    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579762    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d915f962-809c-4394-aa7b-72b49fdde9b2-ovnkube-script-lib\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579787    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-multus-conf-dir\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579806    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-cnibin\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.580032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579822    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dd3cac63-b412-453b-90bc-5b7897a9920f-serviceca\") pod \"node-ca-fng7l\" (UID: \"dd3cac63-b412-453b-90bc-5b7897a9920f\") " pod="openshift-image-registry/node-ca-fng7l"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579838    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-registration-dir\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579876    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-var-lib-kubelet\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579899    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-host\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579914    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-var-lib-openvswitch\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579942    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-log-socket\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579973    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d915f962-809c-4394-aa7b-72b49fdde9b2-ovnkube-config\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.579992    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-var-lib-cni-bin\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580023    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-systemd\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580042    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-sys\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580056    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-lib-modules\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580088    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-etc-openvswitch\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580103    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-cnibin\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580119    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bsgk\" (UniqueName: \"kubernetes.io/projected/f18ac135-e44e-413a-9bda-248eacb6287b-kube-api-access-7bsgk\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580140    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-cni-bin\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580169    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9g84\" (UniqueName: \"kubernetes.io/projected/d915f962-809c-4394-aa7b-72b49fdde9b2-kube-api-access-p9g84\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.580481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580190    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/24b56ac3-1415-4f69-aedc-44c6c2efd7dc-konnectivity-ca\") pod \"konnectivity-agent-g8hbm\" (UID: \"24b56ac3-1415-4f69-aedc-44c6c2efd7dc\") " pod="kube-system/konnectivity-agent-g8hbm"
Jul 03 09:43:19.580945 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580207    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.580945 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580229    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-run-systemd\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.580945 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580252    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-multus-cni-dir\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.580945 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580280    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-run-netns\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.580945 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580299    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-socket-dir\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.580945 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580325    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-kubernetes\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.580945 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580340    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-etc-kubernetes\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.580945 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580355    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-system-cni-dir\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.580945 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580370    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.580945 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580384    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20bc336f-ff0c-4072-8275-46857fb9f016-tmp-dir\") pod \"node-resolver-gmjss\" (UID: \"20bc336f-ff0c-4072-8275-46857fb9f016\") " pod="openshift-dns/node-resolver-gmjss"
Jul 03 09:43:19.580945 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580398    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-os-release\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.580945 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.580422    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mkbj\" (UniqueName: \"kubernetes.io/projected/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-kube-api-access-2mkbj\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.585981 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.585964    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 03 09:43:19.586050 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.586039    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-129.ec2.internal"
Jul 03 09:43:19.587695 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.587679    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal"]
Jul 03 09:43:19.591014 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.590995    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 03 09:43:19.595511 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.595495    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-129.ec2.internal"]
Jul 03 09:43:19.595596 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.595528    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 03 09:43:19.618441 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.618398    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ckpwf"
Jul 03 09:43:19.627052 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.627033    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ckpwf"
Jul 03 09:43:19.673825 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.673802    2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 03 09:43:19.681174 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681153    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.681290 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681181    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-modprobe-d\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.681290 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681200    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-sysctl-conf\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.681290 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681218    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-run-ovn\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.681290 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681239    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/20bc336f-ff0c-4072-8275-46857fb9f016-hosts-file\") pod \"node-resolver-gmjss\" (UID: \"20bc336f-ff0c-4072-8275-46857fb9f016\") " pod="openshift-dns/node-resolver-gmjss"
Jul 03 09:43:19.681519 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681298    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/20bc336f-ff0c-4072-8275-46857fb9f016-hosts-file\") pod \"node-resolver-gmjss\" (UID: \"20bc336f-ff0c-4072-8275-46857fb9f016\") " pod="openshift-dns/node-resolver-gmjss"
Jul 03 09:43:19.681519 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681342    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.681519 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681359    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-modprobe-d\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.681519 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681370    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-sysctl-conf\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.681519 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681374    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.681519 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681404    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-systemd-units\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.681519 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681368    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-systemd-units\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.681519 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681443    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-run-openvswitch\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.681519 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681469    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-os-release\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.681519 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681507    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/420a7e0c-545b-437b-bf1c-6a5e24143c06-multus-daemon-config\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681527    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-run-multus-certs\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681526    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-run-openvswitch\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681551    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-os-release\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681568    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d642348a-9842-43db-bcdf-5d84ef42e828-dbus\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681597    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/24b56ac3-1415-4f69-aedc-44c6c2efd7dc-agent-certs\") pod \"konnectivity-agent-g8hbm\" (UID: \"24b56ac3-1415-4f69-aedc-44c6c2efd7dc\") " pod="kube-system/konnectivity-agent-g8hbm"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681617    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-sysctl-d\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681672    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-run-multus-certs\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681729    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-sysctl-d\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681738    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa4df66b-74e4-4cde-96dc-9ae773748dd8-tmp\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681772    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-multus-socket-dir-parent\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681777    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d642348a-9842-43db-bcdf-5d84ef42e828-dbus\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681802    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrpbg\" (UniqueName: \"kubernetes.io/projected/a70ff444-3447-4dc7-aafb-d91e4f620ec1-kube-api-access-mrpbg\") pod \"iptables-alerter-z7ljw\" (UID: \"a70ff444-3447-4dc7-aafb-d91e4f620ec1\") " pod="openshift-network-operator/iptables-alerter-z7ljw"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681834    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-device-dir\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681874    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-run\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681902    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlwt4\" (UniqueName: \"kubernetes.io/projected/aa4df66b-74e4-4cde-96dc-9ae773748dd8-kube-api-access-dlwt4\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681930    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.682061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681959    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-var-lib-cni-multus\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681969    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-multus-socket-dir-parent\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.681987    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682035    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682062    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-sysconfig\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682092    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-run-netns\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682119    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-node-log\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682141    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-device-dir\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682147    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d915f962-809c-4394-aa7b-72b49fdde9b2-ovn-node-metrics-cert\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682179    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-run\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682179    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/420a7e0c-545b-437b-bf1c-6a5e24143c06-multus-daemon-config\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682215    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682182    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682241    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-sysconfig\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.682256    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682089    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682263    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/420a7e0c-545b-437b-bf1c-6a5e24143c06-cni-binary-copy\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682302    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-hostroot\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.682966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682328    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-sys-fs\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.682368    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs podName:f18ac135-e44e-413a-9bda-248eacb6287b nodeName:}" failed. No retries permitted until 2026-07-03 09:43:20.182322658 +0000 UTC m=+2.047611310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs") pod "network-metrics-daemon-txhn2" (UID: "f18ac135-e44e-413a-9bda-248eacb6287b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682377    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-sys-fs\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.682383    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682409    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-tuned\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682419    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-hostroot\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682410    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682116    2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682441    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-var-lib-cni-multus\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682434    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-node-log\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.682462    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret podName:d642348a-9842-43db-bcdf-5d84ef42e828 nodeName:}" failed. No retries permitted until 2026-07-03 09:43:20.182428351 +0000 UTC m=+2.047717001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret") pod "global-pull-secret-syncer-n4gfv" (UID: "d642348a-9842-43db-bcdf-5d84ef42e828") : object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682481    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-run-ovn\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682492    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-kubelet\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682526    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-kubelet\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682577    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-run-netns\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682658    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-cni-netd\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682739    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/420a7e0c-545b-437b-bf1c-6a5e24143c06-cni-binary-copy\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.683927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682608    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-cni-netd\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682850    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-run-k8s-cni-cncf-io\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682880    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-slash\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682907    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d642348a-9842-43db-bcdf-5d84ef42e828-kubelet-config\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682931    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd3cac63-b412-453b-90bc-5b7897a9920f-host\") pod \"node-ca-fng7l\" (UID: \"dd3cac63-b412-453b-90bc-5b7897a9920f\") " pod="openshift-image-registry/node-ca-fng7l"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682957    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-system-cni-dir\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.682994    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683026    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78fs7\" (UniqueName: \"kubernetes.io/projected/6328c973-29b4-40b7-8467-7a1439e175bc-kube-api-access-78fs7\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683064    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d915f962-809c-4394-aa7b-72b49fdde9b2-env-overrides\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683095    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkm6g\" (UniqueName: \"kubernetes.io/projected/dd3cac63-b412-453b-90bc-5b7897a9920f-kube-api-access-lkm6g\") pod \"node-ca-fng7l\" (UID: \"dd3cac63-b412-453b-90bc-5b7897a9920f\") " pod="openshift-image-registry/node-ca-fng7l"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683121    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wzc7\" (UniqueName: \"kubernetes.io/projected/20bc336f-ff0c-4072-8275-46857fb9f016-kube-api-access-6wzc7\") pod \"node-resolver-gmjss\" (UID: \"20bc336f-ff0c-4072-8275-46857fb9f016\") " pod="openshift-dns/node-resolver-gmjss"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683149    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-var-lib-kubelet\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683174    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdk7m\" (UniqueName: \"kubernetes.io/projected/420a7e0c-545b-437b-bf1c-6a5e24143c06-kube-api-access-zdk7m\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683200    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683226    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d915f962-809c-4394-aa7b-72b49fdde9b2-ovnkube-script-lib\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683251    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-multus-conf-dir\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683276    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-cnibin\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.685031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683300    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dd3cac63-b412-453b-90bc-5b7897a9920f-serviceca\") pod \"node-ca-fng7l\" (UID: \"dd3cac63-b412-453b-90bc-5b7897a9920f\") " pod="openshift-image-registry/node-ca-fng7l"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683353    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-registration-dir\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683384    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-var-lib-kubelet\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683441    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-host\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683467    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-var-lib-openvswitch\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683495    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-log-socket\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683522    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d915f962-809c-4394-aa7b-72b49fdde9b2-ovnkube-config\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683560    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-var-lib-cni-bin\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683586    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-systemd\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683610    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-sys\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683635    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-lib-modules\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683697    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-etc-openvswitch\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683726    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-cnibin\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683757    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a70ff444-3447-4dc7-aafb-d91e4f620ec1-iptables-alerter-script\") pod \"iptables-alerter-z7ljw\" (UID: \"a70ff444-3447-4dc7-aafb-d91e4f620ec1\") " pod="openshift-network-operator/iptables-alerter-z7ljw"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683787    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bsgk\" (UniqueName: \"kubernetes.io/projected/f18ac135-e44e-413a-9bda-248eacb6287b-kube-api-access-7bsgk\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683824    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-cni-bin\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683854    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9g84\" (UniqueName: \"kubernetes.io/projected/d915f962-809c-4394-aa7b-72b49fdde9b2-kube-api-access-p9g84\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.685934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683883    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgll7\" (UniqueName: \"kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7\") pod \"network-check-target-d6fcm\" (UID: \"1e886875-0734-4d9e-9ec5-da0aeeefadda\") " pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683910    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/24b56ac3-1415-4f69-aedc-44c6c2efd7dc-konnectivity-ca\") pod \"konnectivity-agent-g8hbm\" (UID: \"24b56ac3-1415-4f69-aedc-44c6c2efd7dc\") " pod="kube-system/konnectivity-agent-g8hbm"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683958    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.683984    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-run-systemd\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684012    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-multus-cni-dir\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684038    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-run-netns\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684073    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a70ff444-3447-4dc7-aafb-d91e4f620ec1-host-slash\") pod \"iptables-alerter-z7ljw\" (UID: \"a70ff444-3447-4dc7-aafb-d91e4f620ec1\") " pod="openshift-network-operator/iptables-alerter-z7ljw"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684100    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-socket-dir\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684129    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-kubernetes\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684156    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-etc-kubernetes\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684184    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-system-cni-dir\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684213    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684241    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20bc336f-ff0c-4072-8275-46857fb9f016-tmp-dir\") pod \"node-resolver-gmjss\" (UID: \"20bc336f-ff0c-4072-8275-46857fb9f016\") " pod="openshift-dns/node-resolver-gmjss"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684268    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-os-release\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684296    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mkbj\" (UniqueName: \"kubernetes.io/projected/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-kube-api-access-2mkbj\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684519    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-var-lib-cni-bin\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684590    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-run-k8s-cni-cncf-io\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.687313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684638    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-systemd\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684650    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-slash\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684699    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd3cac63-b412-453b-90bc-5b7897a9920f-host\") pod \"node-ca-fng7l\" (UID: \"dd3cac63-b412-453b-90bc-5b7897a9920f\") " pod="openshift-image-registry/node-ca-fng7l"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684750    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-lib-modules\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684769    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d642348a-9842-43db-bcdf-5d84ef42e828-kubelet-config\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684803    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-etc-openvswitch\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684854    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-system-cni-dir\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684848    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-run-netns\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.684870    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-multus-cni-dir\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.685017    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-sys\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.685049    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-cnibin\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.685315    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-socket-dir\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.685326    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-host\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.685373    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-var-lib-openvswitch\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.685378    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.685384    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-kubernetes\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.685422    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.685459    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-host-cni-bin\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.687969 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.685459    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-etc-kubernetes\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.685504    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-multus-conf-dir\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.685998    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-log-socket\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686048    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686087    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686094    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d915f962-809c-4394-aa7b-72b49fdde9b2-env-overrides\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686104    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d915f962-809c-4394-aa7b-72b49fdde9b2-ovnkube-script-lib\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686169    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-system-cni-dir\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686211    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-cnibin\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686258    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d915f962-809c-4394-aa7b-72b49fdde9b2-run-systemd\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686308    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6328c973-29b4-40b7-8467-7a1439e175bc-registration-dir\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686355    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa4df66b-74e4-4cde-96dc-9ae773748dd8-var-lib-kubelet\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686400    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/24b56ac3-1415-4f69-aedc-44c6c2efd7dc-konnectivity-ca\") pod \"konnectivity-agent-g8hbm\" (UID: \"24b56ac3-1415-4f69-aedc-44c6c2efd7dc\") " pod="kube-system/konnectivity-agent-g8hbm"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686406    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20bc336f-ff0c-4072-8275-46857fb9f016-tmp-dir\") pod \"node-resolver-gmjss\" (UID: \"20bc336f-ff0c-4072-8275-46857fb9f016\") " pod="openshift-dns/node-resolver-gmjss"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686455    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-os-release\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686580    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/24b56ac3-1415-4f69-aedc-44c6c2efd7dc-agent-certs\") pod \"konnectivity-agent-g8hbm\" (UID: \"24b56ac3-1415-4f69-aedc-44c6c2efd7dc\") " pod="kube-system/konnectivity-agent-g8hbm"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686606    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dd3cac63-b412-453b-90bc-5b7897a9920f-serviceca\") pod \"node-ca-fng7l\" (UID: \"dd3cac63-b412-453b-90bc-5b7897a9920f\") " pod="openshift-image-registry/node-ca-fng7l"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686664    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/420a7e0c-545b-437b-bf1c-6a5e24143c06-host-var-lib-kubelet\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.688516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.686979    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa4df66b-74e4-4cde-96dc-9ae773748dd8-etc-tuned\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.689145 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.687118    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d915f962-809c-4394-aa7b-72b49fdde9b2-ovnkube-config\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.689145 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.687578    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d915f962-809c-4394-aa7b-72b49fdde9b2-ovn-node-metrics-cert\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.689145 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.687698    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa4df66b-74e4-4cde-96dc-9ae773748dd8-tmp\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.695352 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.695325    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlwt4\" (UniqueName: \"kubernetes.io/projected/aa4df66b-74e4-4cde-96dc-9ae773748dd8-kube-api-access-dlwt4\") pod \"tuned-r89ht\" (UID: \"aa4df66b-74e4-4cde-96dc-9ae773748dd8\") " pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.696034 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.696007    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mkbj\" (UniqueName: \"kubernetes.io/projected/785d93fe-f1db-45ab-95b7-4e0d4ccf20fb-kube-api-access-2mkbj\") pod \"multus-additional-cni-plugins-dtsgm\" (UID: \"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb\") " pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.697425 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.697398    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78fs7\" (UniqueName: \"kubernetes.io/projected/6328c973-29b4-40b7-8467-7a1439e175bc-kube-api-access-78fs7\") pod \"aws-ebs-csi-driver-node-hskrg\" (UID: \"6328c973-29b4-40b7-8467-7a1439e175bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.697760 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.697724    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9g84\" (UniqueName: \"kubernetes.io/projected/d915f962-809c-4394-aa7b-72b49fdde9b2-kube-api-access-p9g84\") pod \"ovnkube-node-8hmsd\" (UID: \"d915f962-809c-4394-aa7b-72b49fdde9b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.697834 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.697811    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wzc7\" (UniqueName: \"kubernetes.io/projected/20bc336f-ff0c-4072-8275-46857fb9f016-kube-api-access-6wzc7\") pod \"node-resolver-gmjss\" (UID: \"20bc336f-ff0c-4072-8275-46857fb9f016\") " pod="openshift-dns/node-resolver-gmjss"
Jul 03 09:43:19.698264 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.698233    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkm6g\" (UniqueName: \"kubernetes.io/projected/dd3cac63-b412-453b-90bc-5b7897a9920f-kube-api-access-lkm6g\") pod \"node-ca-fng7l\" (UID: \"dd3cac63-b412-453b-90bc-5b7897a9920f\") " pod="openshift-image-registry/node-ca-fng7l"
Jul 03 09:43:19.698349 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.698233    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bsgk\" (UniqueName: \"kubernetes.io/projected/f18ac135-e44e-413a-9bda-248eacb6287b-kube-api-access-7bsgk\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:19.699783 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.699756    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdk7m\" (UniqueName: \"kubernetes.io/projected/420a7e0c-545b-437b-bf1c-6a5e24143c06-kube-api-access-zdk7m\") pod \"multus-4pl6g\" (UID: \"420a7e0c-545b-437b-bf1c-6a5e24143c06\") " pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.711965 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.711790    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fng7l"
Jul 03 09:43:19.785230 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.785193    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrpbg\" (UniqueName: \"kubernetes.io/projected/a70ff444-3447-4dc7-aafb-d91e4f620ec1-kube-api-access-mrpbg\") pod \"iptables-alerter-z7ljw\" (UID: \"a70ff444-3447-4dc7-aafb-d91e4f620ec1\") " pod="openshift-network-operator/iptables-alerter-z7ljw"
Jul 03 09:43:19.785402 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.785293    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a70ff444-3447-4dc7-aafb-d91e4f620ec1-iptables-alerter-script\") pod \"iptables-alerter-z7ljw\" (UID: \"a70ff444-3447-4dc7-aafb-d91e4f620ec1\") " pod="openshift-network-operator/iptables-alerter-z7ljw"
Jul 03 09:43:19.785402 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.785312    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgll7\" (UniqueName: \"kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7\") pod \"network-check-target-d6fcm\" (UID: \"1e886875-0734-4d9e-9ec5-da0aeeefadda\") " pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:19.785402 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.785332    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a70ff444-3447-4dc7-aafb-d91e4f620ec1-host-slash\") pod \"iptables-alerter-z7ljw\" (UID: \"a70ff444-3447-4dc7-aafb-d91e4f620ec1\") " pod="openshift-network-operator/iptables-alerter-z7ljw"
Jul 03 09:43:19.785563 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.785401    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a70ff444-3447-4dc7-aafb-d91e4f620ec1-host-slash\") pod \"iptables-alerter-z7ljw\" (UID: \"a70ff444-3447-4dc7-aafb-d91e4f620ec1\") " pod="openshift-network-operator/iptables-alerter-z7ljw"
Jul 03 09:43:19.785948 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.785923    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a70ff444-3447-4dc7-aafb-d91e4f620ec1-iptables-alerter-script\") pod \"iptables-alerter-z7ljw\" (UID: \"a70ff444-3447-4dc7-aafb-d91e4f620ec1\") " pod="openshift-network-operator/iptables-alerter-z7ljw"
Jul 03 09:43:19.793589 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.793565    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 09:43:19.793691 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.793593    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 09:43:19.793691 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.793610    2571 projected.go:196] Error preparing data for projected volume kube-api-access-pgll7 for pod openshift-network-diagnostics/network-check-target-d6fcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:19.793691 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:19.793680    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7 podName:1e886875-0734-4d9e-9ec5-da0aeeefadda nodeName:}" failed. No retries permitted until 2026-07-03 09:43:20.293660471 +0000 UTC m=+2.158949110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pgll7" (UniqueName: "kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7") pod "network-check-target-d6fcm" (UID: "1e886875-0734-4d9e-9ec5-da0aeeefadda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:19.796266 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.796245    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrpbg\" (UniqueName: \"kubernetes.io/projected/a70ff444-3447-4dc7-aafb-d91e4f620ec1-kube-api-access-mrpbg\") pod \"iptables-alerter-z7ljw\" (UID: \"a70ff444-3447-4dc7-aafb-d91e4f620ec1\") " pod="openshift-network-operator/iptables-alerter-z7ljw"
Jul 03 09:43:19.803258 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:19.803216    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f9c3a7f49c697f26f22966d3293ba5.slice/crio-c96b307d2c29f8d986eb29a9a74c0f8a81fc57e4ab6c4411beb63dd670a4abe3 WatchSource:0}: Error finding container c96b307d2c29f8d986eb29a9a74c0f8a81fc57e4ab6c4411beb63dd670a4abe3: Status 404 returned error can't find the container with id c96b307d2c29f8d986eb29a9a74c0f8a81fc57e4ab6c4411beb63dd670a4abe3
Jul 03 09:43:19.803582 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:19.803556    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb662888df2876571019936bda200019b.slice/crio-16778fa2bc22549ec027fee5674772fbdbb883ade79b20e7582261a37ab81e8b WatchSource:0}: Error finding container 16778fa2bc22549ec027fee5674772fbdbb883ade79b20e7582261a37ab81e8b: Status 404 returned error can't find the container with id 16778fa2bc22549ec027fee5674772fbdbb883ade79b20e7582261a37ab81e8b
Jul 03 09:43:19.808343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.808328    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 09:43:19.853284 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.853254    2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 09:43:19.890047 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.890019    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:19.900656 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:19.900624    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd915f962_809c_4394_aa7b_72b49fdde9b2.slice/crio-7e9acb68338c0ca70f23855cd158ebbc88722616067ad32c35cff225468ae711 WatchSource:0}: Error finding container 7e9acb68338c0ca70f23855cd158ebbc88722616067ad32c35cff225468ae711: Status 404 returned error can't find the container with id 7e9acb68338c0ca70f23855cd158ebbc88722616067ad32c35cff225468ae711
Jul 03 09:43:19.905105 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.905060    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g8hbm"
Jul 03 09:43:19.914112 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:19.914085    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b56ac3_1415_4f69_aedc_44c6c2efd7dc.slice/crio-a6f1c2c32007db5378a8be98edb7d85bc1612c643a8b8528cc81840dba5ca480 WatchSource:0}: Error finding container a6f1c2c32007db5378a8be98edb7d85bc1612c643a8b8528cc81840dba5ca480: Status 404 returned error can't find the container with id a6f1c2c32007db5378a8be98edb7d85bc1612c643a8b8528cc81840dba5ca480
Jul 03 09:43:19.927411 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.927385    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gmjss"
Jul 03 09:43:19.936988 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:19.936965    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20bc336f_ff0c_4072_8275_46857fb9f016.slice/crio-e9b41e04659faaefee1f670bada8761fac1ac24b64eb04752b390c6ca6ba6ad3 WatchSource:0}: Error finding container e9b41e04659faaefee1f670bada8761fac1ac24b64eb04752b390c6ca6ba6ad3: Status 404 returned error can't find the container with id e9b41e04659faaefee1f670bada8761fac1ac24b64eb04752b390c6ca6ba6ad3
Jul 03 09:43:19.942158 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.942134    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4pl6g"
Jul 03 09:43:19.946233 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.946210    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dtsgm"
Jul 03 09:43:19.952890 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:19.952866    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod420a7e0c_545b_437b_bf1c_6a5e24143c06.slice/crio-3b3e55f9c8c7c699e4f7871e13541c88cb2554adc601da991afe902788ce233d WatchSource:0}: Error finding container 3b3e55f9c8c7c699e4f7871e13541c88cb2554adc601da991afe902788ce233d: Status 404 returned error can't find the container with id 3b3e55f9c8c7c699e4f7871e13541c88cb2554adc601da991afe902788ce233d
Jul 03 09:43:19.955782 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:19.955754    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod785d93fe_f1db_45ab_95b7_4e0d4ccf20fb.slice/crio-e25962e245d25b2853324da73acefca88d112c463c31bc037e9fee01d775f342 WatchSource:0}: Error finding container e25962e245d25b2853324da73acefca88d112c463c31bc037e9fee01d775f342: Status 404 returned error can't find the container with id e25962e245d25b2853324da73acefca88d112c463c31bc037e9fee01d775f342
Jul 03 09:43:19.972468 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.972446    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg"
Jul 03 09:43:19.979661 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:19.979639    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r89ht"
Jul 03 09:43:19.982726 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:19.982705    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6328c973_29b4_40b7_8467_7a1439e175bc.slice/crio-6401897a63ef9cdfe46984cccc52b55029bbe89aa084f86f1ef3865871af8963 WatchSource:0}: Error finding container 6401897a63ef9cdfe46984cccc52b55029bbe89aa084f86f1ef3865871af8963: Status 404 returned error can't find the container with id 6401897a63ef9cdfe46984cccc52b55029bbe89aa084f86f1ef3865871af8963
Jul 03 09:43:19.988492 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:19.988463    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa4df66b_74e4_4cde_96dc_9ae773748dd8.slice/crio-893baadb5d312c782fa7d5fefd088098a0f52703fe4e8c1a21d5c03ef6672f61 WatchSource:0}: Error finding container 893baadb5d312c782fa7d5fefd088098a0f52703fe4e8c1a21d5c03ef6672f61: Status 404 returned error can't find the container with id 893baadb5d312c782fa7d5fefd088098a0f52703fe4e8c1a21d5c03ef6672f61
Jul 03 09:43:20.020122 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.020085    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z7ljw"
Jul 03 09:43:20.030215 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:20.030192    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda70ff444_3447_4dc7_aafb_d91e4f620ec1.slice/crio-9f379ca51a3f46deb4596504252e83e0059484b0577fda782864f52cb64e9ea5 WatchSource:0}: Error finding container 9f379ca51a3f46deb4596504252e83e0059484b0577fda782864f52cb64e9ea5: Status 404 returned error can't find the container with id 9f379ca51a3f46deb4596504252e83e0059484b0577fda782864f52cb64e9ea5
Jul 03 09:43:20.062035 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:20.062003    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd3cac63_b412_453b_90bc_5b7897a9920f.slice/crio-153dc4c6b16c4391dafe9808143adea06e3912cd287325f59b88426a3305997c WatchSource:0}: Error finding container 153dc4c6b16c4391dafe9808143adea06e3912cd287325f59b88426a3305997c: Status 404 returned error can't find the container with id 153dc4c6b16c4391dafe9808143adea06e3912cd287325f59b88426a3305997c
Jul 03 09:43:20.190325 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.189096    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:20.190325 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.189151    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:20.190325 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:20.189299    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:20.190325 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:20.189365    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs podName:f18ac135-e44e-413a-9bda-248eacb6287b nodeName:}" failed. No retries permitted until 2026-07-03 09:43:21.189343761 +0000 UTC m=+3.054632410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs") pod "network-metrics-daemon-txhn2" (UID: "f18ac135-e44e-413a-9bda-248eacb6287b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:20.190325 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:20.189781    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:20.190325 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:20.189833    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret podName:d642348a-9842-43db-bcdf-5d84ef42e828 nodeName:}" failed. No retries permitted until 2026-07-03 09:43:21.189817813 +0000 UTC m=+3.055106455 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret") pod "global-pull-secret-syncer-n4gfv" (UID: "d642348a-9842-43db-bcdf-5d84ef42e828") : object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:20.390893 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.390825    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgll7\" (UniqueName: \"kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7\") pod \"network-check-target-d6fcm\" (UID: \"1e886875-0734-4d9e-9ec5-da0aeeefadda\") " pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:20.391077 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:20.391046    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 09:43:20.391077 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:20.391071    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 09:43:20.391202 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:20.391085    2571 projected.go:196] Error preparing data for projected volume kube-api-access-pgll7 for pod openshift-network-diagnostics/network-check-target-d6fcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:20.391202 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:20.391159    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7 podName:1e886875-0734-4d9e-9ec5-da0aeeefadda nodeName:}" failed. No retries permitted until 2026-07-03 09:43:21.39113753 +0000 UTC m=+3.256426169 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgll7" (UniqueName: "kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7") pod "network-check-target-d6fcm" (UID: "1e886875-0734-4d9e-9ec5-da0aeeefadda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:20.618860 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.618759    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 09:43:20.629027 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.628982    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 09:38:20 +0000 UTC" deadline="2027-12-28 06:43:13.841089089 +0000 UTC"
Jul 03 09:43:20.629027 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.629022    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13028h59m53.212072084s"
Jul 03 09:43:20.749852 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.749802    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 09:43:20.758972 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.758944    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:20.759126 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:20.759094    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:20.759668 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.759634    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:20.759787 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:20.759762    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:20.776789 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.776728    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fng7l" event={"ID":"dd3cac63-b412-453b-90bc-5b7897a9920f","Type":"ContainerStarted","Data":"153dc4c6b16c4391dafe9808143adea06e3912cd287325f59b88426a3305997c"}
Jul 03 09:43:20.781270 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.781196    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z7ljw" event={"ID":"a70ff444-3447-4dc7-aafb-d91e4f620ec1","Type":"ContainerStarted","Data":"9f379ca51a3f46deb4596504252e83e0059484b0577fda782864f52cb64e9ea5"}
Jul 03 09:43:20.796762 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.796699    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg" event={"ID":"6328c973-29b4-40b7-8467-7a1439e175bc","Type":"ContainerStarted","Data":"6401897a63ef9cdfe46984cccc52b55029bbe89aa084f86f1ef3865871af8963"}
Jul 03 09:43:20.806989 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.806950    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gmjss" event={"ID":"20bc336f-ff0c-4072-8275-46857fb9f016","Type":"ContainerStarted","Data":"e9b41e04659faaefee1f670bada8761fac1ac24b64eb04752b390c6ca6ba6ad3"}
Jul 03 09:43:20.815479 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.815442    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd" event={"ID":"d915f962-809c-4394-aa7b-72b49fdde9b2","Type":"ContainerStarted","Data":"7e9acb68338c0ca70f23855cd158ebbc88722616067ad32c35cff225468ae711"}
Jul 03 09:43:20.819760 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.819725    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal" event={"ID":"d6f9c3a7f49c697f26f22966d3293ba5","Type":"ContainerStarted","Data":"c96b307d2c29f8d986eb29a9a74c0f8a81fc57e4ab6c4411beb63dd670a4abe3"}
Jul 03 09:43:20.828933 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.828903    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r89ht" event={"ID":"aa4df66b-74e4-4cde-96dc-9ae773748dd8","Type":"ContainerStarted","Data":"893baadb5d312c782fa7d5fefd088098a0f52703fe4e8c1a21d5c03ef6672f61"}
Jul 03 09:43:20.840835 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.840807    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtsgm" event={"ID":"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb","Type":"ContainerStarted","Data":"e25962e245d25b2853324da73acefca88d112c463c31bc037e9fee01d775f342"}
Jul 03 09:43:20.859621 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.859586    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4pl6g" event={"ID":"420a7e0c-545b-437b-bf1c-6a5e24143c06","Type":"ContainerStarted","Data":"3b3e55f9c8c7c699e4f7871e13541c88cb2554adc601da991afe902788ce233d"}
Jul 03 09:43:20.862991 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.862954    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g8hbm" event={"ID":"24b56ac3-1415-4f69-aedc-44c6c2efd7dc","Type":"ContainerStarted","Data":"a6f1c2c32007db5378a8be98edb7d85bc1612c643a8b8528cc81840dba5ca480"}
Jul 03 09:43:20.873363 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:20.873265    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-129.ec2.internal" event={"ID":"b662888df2876571019936bda200019b","Type":"ContainerStarted","Data":"16778fa2bc22549ec027fee5674772fbdbb883ade79b20e7582261a37ab81e8b"}
Jul 03 09:43:21.197944 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:21.197826    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:21.197944 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:21.197899    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:21.198182 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:21.198081    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:21.198182 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:21.198144    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs podName:f18ac135-e44e-413a-9bda-248eacb6287b nodeName:}" failed. No retries permitted until 2026-07-03 09:43:23.198123824 +0000 UTC m=+5.063412463 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs") pod "network-metrics-daemon-txhn2" (UID: "f18ac135-e44e-413a-9bda-248eacb6287b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:21.198586 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:21.198550    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:21.198659 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:21.198611    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret podName:d642348a-9842-43db-bcdf-5d84ef42e828 nodeName:}" failed. No retries permitted until 2026-07-03 09:43:23.198591117 +0000 UTC m=+5.063879752 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret") pod "global-pull-secret-syncer-n4gfv" (UID: "d642348a-9842-43db-bcdf-5d84ef42e828") : object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:21.402929 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:21.401307    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgll7\" (UniqueName: \"kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7\") pod \"network-check-target-d6fcm\" (UID: \"1e886875-0734-4d9e-9ec5-da0aeeefadda\") " pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:21.402929 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:21.401526    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 09:43:21.402929 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:21.401551    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 09:43:21.402929 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:21.401565    2571 projected.go:196] Error preparing data for projected volume kube-api-access-pgll7 for pod openshift-network-diagnostics/network-check-target-d6fcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:21.402929 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:21.401628    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7 podName:1e886875-0734-4d9e-9ec5-da0aeeefadda nodeName:}" failed. No retries permitted until 2026-07-03 09:43:23.401608298 +0000 UTC m=+5.266896939 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgll7" (UniqueName: "kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7") pod "network-check-target-d6fcm" (UID: "1e886875-0734-4d9e-9ec5-da0aeeefadda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:21.629768 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:21.629601    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 09:38:20 +0000 UTC" deadline="2028-04-16 15:47:03.085554694 +0000 UTC"
Jul 03 09:43:21.629768 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:21.629661    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15678h3m41.455897224s"
Jul 03 09:43:21.759067 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:21.759031    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:21.759329 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:21.759182    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:22.761200 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:22.761065    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:22.761830 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:22.761225    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:22.761830 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:22.761592    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:22.761830 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:22.761684    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:23.217848 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:23.217772    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:23.217848 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:23.217816    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:23.218142 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:23.217973    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:23.218142 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:23.218096    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:23.218263 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:23.218146    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret podName:d642348a-9842-43db-bcdf-5d84ef42e828 nodeName:}" failed. No retries permitted until 2026-07-03 09:43:27.218127261 +0000 UTC m=+9.083415901 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret") pod "global-pull-secret-syncer-n4gfv" (UID: "d642348a-9842-43db-bcdf-5d84ef42e828") : object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:23.218591 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:23.218555    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs podName:f18ac135-e44e-413a-9bda-248eacb6287b nodeName:}" failed. No retries permitted until 2026-07-03 09:43:27.218531295 +0000 UTC m=+9.083819944 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs") pod "network-metrics-daemon-txhn2" (UID: "f18ac135-e44e-413a-9bda-248eacb6287b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:23.419800 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:23.418889    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgll7\" (UniqueName: \"kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7\") pod \"network-check-target-d6fcm\" (UID: \"1e886875-0734-4d9e-9ec5-da0aeeefadda\") " pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:23.419800 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:23.419071    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 09:43:23.419800 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:23.419092    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 09:43:23.419800 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:23.419106    2571 projected.go:196] Error preparing data for projected volume kube-api-access-pgll7 for pod openshift-network-diagnostics/network-check-target-d6fcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:23.419800 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:23.419171    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7 podName:1e886875-0734-4d9e-9ec5-da0aeeefadda nodeName:}" failed. No retries permitted until 2026-07-03 09:43:27.419144337 +0000 UTC m=+9.284432975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgll7" (UniqueName: "kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7") pod "network-check-target-d6fcm" (UID: "1e886875-0734-4d9e-9ec5-da0aeeefadda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:23.758031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:23.758000    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:23.758217 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:23.758124    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:24.761020 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:24.760987    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:24.761521 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:24.760987    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:24.761521 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:24.761154    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:24.761521 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:24.761303    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:25.759104 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:25.758839    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:25.759104 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:25.759005    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:26.758577 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:26.758544    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:26.759084 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:26.758544    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:26.759084 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:26.758738    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:26.759084 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:26.758883    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:27.256054 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:27.256010    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:27.256255 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:27.256072    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:27.256255 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:27.256141    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:27.256255 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:27.256167    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:27.256255 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:27.256213    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret podName:d642348a-9842-43db-bcdf-5d84ef42e828 nodeName:}" failed. No retries permitted until 2026-07-03 09:43:35.25619357 +0000 UTC m=+17.121482214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret") pod "global-pull-secret-syncer-n4gfv" (UID: "d642348a-9842-43db-bcdf-5d84ef42e828") : object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:27.256255 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:27.256235    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs podName:f18ac135-e44e-413a-9bda-248eacb6287b nodeName:}" failed. No retries permitted until 2026-07-03 09:43:35.256224078 +0000 UTC m=+17.121512713 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs") pod "network-metrics-daemon-txhn2" (UID: "f18ac135-e44e-413a-9bda-248eacb6287b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:27.457377 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:27.457330    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgll7\" (UniqueName: \"kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7\") pod \"network-check-target-d6fcm\" (UID: \"1e886875-0734-4d9e-9ec5-da0aeeefadda\") " pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:27.457555 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:27.457516    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 09:43:27.457555 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:27.457547    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 09:43:27.457700 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:27.457562    2571 projected.go:196] Error preparing data for projected volume kube-api-access-pgll7 for pod openshift-network-diagnostics/network-check-target-d6fcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:27.457700 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:27.457633    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7 podName:1e886875-0734-4d9e-9ec5-da0aeeefadda nodeName:}" failed. No retries permitted until 2026-07-03 09:43:35.457612386 +0000 UTC m=+17.322901024 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgll7" (UniqueName: "kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7") pod "network-check-target-d6fcm" (UID: "1e886875-0734-4d9e-9ec5-da0aeeefadda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:27.759522 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:27.759030    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:27.759522 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:27.759183    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:28.760133 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:28.759443    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:28.760133 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:28.759586    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:28.760133 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:28.759996    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:28.760133 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:28.760091    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:29.758220 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:29.758153    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:29.758401 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:29.758282    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:30.758090 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:30.758040    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:30.758090 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:30.758090    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:30.758597 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:30.758187    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:30.758597 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:30.758308    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:31.758662 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:31.758622    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:31.759238 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:31.758750    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:32.758204 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:32.758165    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:32.758383 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:32.758218    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:32.758383 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:32.758296    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:32.758493 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:32.758430    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:33.758704 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:33.758672    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:33.759147 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:33.758785    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:34.759053 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:34.759014    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:34.759053 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:34.759043    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:34.759571 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:34.759152    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:34.759571 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:34.759262    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:35.321823 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:35.321774    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:35.321823 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:35.321821    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:35.322057 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:35.321946    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:35.322057 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:35.321954    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:35.322057 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:35.322004    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs podName:f18ac135-e44e-413a-9bda-248eacb6287b nodeName:}" failed. No retries permitted until 2026-07-03 09:43:51.321985458 +0000 UTC m=+33.187274092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs") pod "network-metrics-daemon-txhn2" (UID: "f18ac135-e44e-413a-9bda-248eacb6287b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:35.322057 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:35.322022    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret podName:d642348a-9842-43db-bcdf-5d84ef42e828 nodeName:}" failed. No retries permitted until 2026-07-03 09:43:51.322012969 +0000 UTC m=+33.187301602 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret") pod "global-pull-secret-syncer-n4gfv" (UID: "d642348a-9842-43db-bcdf-5d84ef42e828") : object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:35.523434 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:35.523391    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgll7\" (UniqueName: \"kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7\") pod \"network-check-target-d6fcm\" (UID: \"1e886875-0734-4d9e-9ec5-da0aeeefadda\") " pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:35.523609 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:35.523578    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 09:43:35.523609 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:35.523604    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 09:43:35.523735 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:35.523617    2571 projected.go:196] Error preparing data for projected volume kube-api-access-pgll7 for pod openshift-network-diagnostics/network-check-target-d6fcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:35.523735 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:35.523685    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7 podName:1e886875-0734-4d9e-9ec5-da0aeeefadda nodeName:}" failed. No retries permitted until 2026-07-03 09:43:51.523665874 +0000 UTC m=+33.388954524 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgll7" (UniqueName: "kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7") pod "network-check-target-d6fcm" (UID: "1e886875-0734-4d9e-9ec5-da0aeeefadda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:35.758232 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:35.758163    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:35.758380 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:35.758279    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:36.758036 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:36.757996    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:36.758430 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:36.758046    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:36.758430 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:36.758137    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:36.758430 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:36.758253    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:37.758369 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:37.758330    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:37.758816 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:37.758466    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:38.762661 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:38.761655    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:38.762661 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:38.761825    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:38.762661 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:38.762298    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:38.762661 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:38.762411    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:39.758448 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:39.758412    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:39.758640 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:39.758537    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:40.758241 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:40.758056    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:40.758695 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:40.758080    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:40.758695 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:40.758347    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:40.758695 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:40.758373    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:40.919961 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:40.919758    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r89ht" event={"ID":"aa4df66b-74e4-4cde-96dc-9ae773748dd8","Type":"ContainerStarted","Data":"140fe2095f19140a57d8e579be849d6dad6852a2ce24fc7028cedbcc3ef05f4b"}
Jul 03 09:43:40.921040 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:40.921021    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4pl6g" event={"ID":"420a7e0c-545b-437b-bf1c-6a5e24143c06","Type":"ContainerStarted","Data":"dd6410a008675b638b8ef251c3699a7940ec1b478bcf2520c6c9fdbd3b38e9f9"}
Jul 03 09:43:40.922335 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:40.922313    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-129.ec2.internal" event={"ID":"b662888df2876571019936bda200019b","Type":"ContainerStarted","Data":"3e40b03d5f3995cfc95e96c570098a52df1953d30f760dbe0a674eb8052f711c"}
Jul 03 09:43:40.923780 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:40.923765    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 09:43:40.926688 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:40.924332    2571 generic.go:356] "Generic (PLEG): container finished" podID="d915f962-809c-4394-aa7b-72b49fdde9b2" containerID="f239ab00a953d99e50623b7d3244040692de4733720826831df27809cfc94ae6" exitCode=1
Jul 03 09:43:40.926688 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:40.924382    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd" event={"ID":"d915f962-809c-4394-aa7b-72b49fdde9b2","Type":"ContainerStarted","Data":"254d908eaf94ac3460b63bcb345b02a280cc94b90e795e255f43994240146903"}
Jul 03 09:43:40.926688 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:40.924644    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd" event={"ID":"d915f962-809c-4394-aa7b-72b49fdde9b2","Type":"ContainerDied","Data":"f239ab00a953d99e50623b7d3244040692de4733720826831df27809cfc94ae6"}
Jul 03 09:43:40.926688 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:40.924728    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd" event={"ID":"d915f962-809c-4394-aa7b-72b49fdde9b2","Type":"ContainerStarted","Data":"b1f45710b7bd2dab29239b84c15ba26f65ed52d77667548cad5778ef12231b48"}
Jul 03 09:43:40.938937 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:40.938889    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-r89ht" podStartSLOduration=1.43304455 podStartE2EDuration="21.938875771s" totalImagesPullingTime="20.505831221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 09:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:43:40.938576304 +0000 UTC m=+22.803864959" watchObservedRunningTime="2026-07-03 09:43:40.938875771 +0000 UTC m=+22.804164426"
Jul 03 09:43:40.951402 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:40.951363    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-129.ec2.internal" podStartSLOduration=20.951349336 podStartE2EDuration="20.951349336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 09:43:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:43:40.950997623 +0000 UTC m=+22.816286278" watchObservedRunningTime="2026-07-03 09:43:40.951349336 +0000 UTC m=+22.816637991"
Jul 03 09:43:40.967583 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:40.967545    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4pl6g" podStartSLOduration=1.375095862 podStartE2EDuration="21.967532167s" totalImagesPullingTime="20.592436305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 09:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:43:40.967080112 +0000 UTC m=+22.832368768" watchObservedRunningTime="2026-07-03 09:43:40.967532167 +0000 UTC m=+22.832820819"
Jul 03 09:43:41.758056 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.758032    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:41.758172 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:41.758150    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:41.927599 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.927559    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g8hbm" event={"ID":"24b56ac3-1415-4f69-aedc-44c6c2efd7dc","Type":"ContainerStarted","Data":"0c46847bd3a492eb5cbb6589106be9e687e781822c2bd3bf3278904a6186b811"}
Jul 03 09:43:41.928980 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.928957    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fng7l" event={"ID":"dd3cac63-b412-453b-90bc-5b7897a9920f","Type":"ContainerStarted","Data":"c4956f1ae3b82b960e768d65b30f1628ff037eebf85c9cadad8222c8cca686ac"}
Jul 03 09:43:41.930291 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.930265    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z7ljw" event={"ID":"a70ff444-3447-4dc7-aafb-d91e4f620ec1","Type":"ContainerStarted","Data":"3fd46176bca5c235e456772a7f37b686caa5ba5ef120b4c2d7894a7e2b93c741"}
Jul 03 09:43:41.931583 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.931565    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg" event={"ID":"6328c973-29b4-40b7-8467-7a1439e175bc","Type":"ContainerStarted","Data":"6b4714454be98911806a353c3e51649dbd833da77e2e26af24ccfa0eda150f37"}
Jul 03 09:43:41.932764 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.932741    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gmjss" event={"ID":"20bc336f-ff0c-4072-8275-46857fb9f016","Type":"ContainerStarted","Data":"74477d7e26cdcfaf4b1bca5356ddf47555c6620136182cac127e9defe7b31be3"}
Jul 03 09:43:41.935088 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.935073    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 09:43:41.935429 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.935409    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd" event={"ID":"d915f962-809c-4394-aa7b-72b49fdde9b2","Type":"ContainerStarted","Data":"a288142610a2ad98fd4589e4360b18494df78fe569ee2c12e712a2fd7d1acc79"}
Jul 03 09:43:41.935490 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.935469    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd" event={"ID":"d915f962-809c-4394-aa7b-72b49fdde9b2","Type":"ContainerStarted","Data":"fdad43181b40e11621cd402475c1c9a2c91f51c6315598594fff07444dc09542"}
Jul 03 09:43:41.935544 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.935502    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd" event={"ID":"d915f962-809c-4394-aa7b-72b49fdde9b2","Type":"ContainerStarted","Data":"c29151507a1f9e2f7956cb50ea757a0d117184ea44991efc953f85b444ba22c1"}
Jul 03 09:43:41.936661 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.936644    2571 generic.go:356] "Generic (PLEG): container finished" podID="d6f9c3a7f49c697f26f22966d3293ba5" containerID="dbf090d0c8e75505fb684302f2896e6f2a0e535266085fa862657532c4ab4a7f" exitCode=0
Jul 03 09:43:41.936739 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.936701    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal" event={"ID":"d6f9c3a7f49c697f26f22966d3293ba5","Type":"ContainerDied","Data":"dbf090d0c8e75505fb684302f2896e6f2a0e535266085fa862657532c4ab4a7f"}
Jul 03 09:43:41.938155 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.938136    2571 generic.go:356] "Generic (PLEG): container finished" podID="785d93fe-f1db-45ab-95b7-4e0d4ccf20fb" containerID="d8d95d8b22e344f231ce4ea0d552566331a7c60a218ad8b0e06b7061419323d9" exitCode=0
Jul 03 09:43:41.938248 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.938195    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtsgm" event={"ID":"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb","Type":"ContainerDied","Data":"d8d95d8b22e344f231ce4ea0d552566331a7c60a218ad8b0e06b7061419323d9"}
Jul 03 09:43:41.942525 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.942351    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g8hbm" podStartSLOduration=2.389021045 podStartE2EDuration="22.942341118s" totalImagesPullingTime="20.553320073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:43:41.941877132 +0000 UTC m=+23.807165787" watchObservedRunningTime="2026-07-03 09:43:41.942341118 +0000 UTC m=+23.807629772"
Jul 03 09:43:41.974111 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:41.974056    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z7ljw" podStartSLOduration=2.516967382 podStartE2EDuration="22.974038945s" totalImagesPullingTime="20.457071563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 09:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:43:41.973859413 +0000 UTC m=+23.839148060" watchObservedRunningTime="2026-07-03 09:43:41.974038945 +0000 UTC m=+23.839327601"
Jul 03 09:43:42.004498 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:42.004403    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fng7l" podStartSLOduration=2.57906009 podStartE2EDuration="23.004384534s" totalImagesPullingTime="20.425324444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 09:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:43:42.00423934 +0000 UTC m=+23.869527995" watchObservedRunningTime="2026-07-03 09:43:42.004384534 +0000 UTC m=+23.869673190"
Jul 03 09:43:42.004946 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:42.004917    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gmjss" podStartSLOduration=2.454543161 podStartE2EDuration="23.004902924s" totalImagesPullingTime="20.550359763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 09:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:43:41.991289061 +0000 UTC m=+23.856577718" watchObservedRunningTime="2026-07-03 09:43:42.004902924 +0000 UTC m=+23.870191583"
Jul 03 09:43:42.590685 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:42.590659    2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 03 09:43:42.662047 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:42.661962    2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-03T09:43:42.590680753Z","UUID":"e1646d51-bf04-44e4-aab7-cea87ba9311c","Handler":null,"Name":"","Endpoint":""}
Jul 03 09:43:42.663278 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:42.663251    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 03 09:43:42.663369 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:42.663285    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 03 09:43:42.758250 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:42.758217    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:42.758436 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:42.758364    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:42.758495 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:42.758360    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:42.758534 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:42.758499    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:42.942641 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:42.942560    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg" event={"ID":"6328c973-29b4-40b7-8467-7a1439e175bc","Type":"ContainerStarted","Data":"a5e1f74580c938e76247d60ffed91a04f792b8bde9be368703aebdc8fcd6116a"}
Jul 03 09:43:42.944506 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:42.944473    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal" event={"ID":"d6f9c3a7f49c697f26f22966d3293ba5","Type":"ContainerStarted","Data":"6fa29f9a813b856dc77f7de23356ed0e164fb2fc3e791a59e6d4b2156a55d48e"}
Jul 03 09:43:42.961225 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:42.961174    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-129.ec2.internal" podStartSLOduration=22.961160362 podStartE2EDuration="22.961160362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 09:43:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:43:42.960894651 +0000 UTC m=+24.826183308" watchObservedRunningTime="2026-07-03 09:43:42.961160362 +0000 UTC m=+24.826449017"
Jul 03 09:43:43.759003 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:43.758972    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:43.759248 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:43.759092    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:43.950264 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:43.950229    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 09:43:43.950696 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:43.950671    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd" event={"ID":"d915f962-809c-4394-aa7b-72b49fdde9b2","Type":"ContainerStarted","Data":"edd9bb51205460ded68722a25f221fc740b0dd8bdd20f9a0f2ea1e000c343b45"}
Jul 03 09:43:43.952906 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:43.952880    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg" event={"ID":"6328c973-29b4-40b7-8467-7a1439e175bc","Type":"ContainerStarted","Data":"ee74b012cd2cb74ec68f98ac412f07459118b1998182213adb5047a8c7f02991"}
Jul 03 09:43:43.976160 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:43.976103    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hskrg" podStartSLOduration=2.186219717 podStartE2EDuration="24.976083847s" totalImagesPullingTime="22.78986413s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 09:43:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:43:43.975367327 +0000 UTC m=+25.840655981" watchObservedRunningTime="2026-07-03 09:43:43.976083847 +0000 UTC m=+25.841372504"
Jul 03 09:43:44.762823 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:44.762794    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:44.763005 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:44.762794    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:44.763005 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:44.762927    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:44.763005 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:44.762969    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:45.758016 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:45.757982    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:45.758414 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:45.758095    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:46.600199 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.600164    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g8hbm"
Jul 03 09:43:46.600962 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.600942    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g8hbm"
Jul 03 09:43:46.758594 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.758510    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:46.759267 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:46.758626    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:46.759267 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.758679    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:46.759267 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:46.758767    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:46.962690 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.962659    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 09:43:46.963069 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.963036    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd" event={"ID":"d915f962-809c-4394-aa7b-72b49fdde9b2","Type":"ContainerStarted","Data":"c75ddec94eb29b7ffa2d5c923944517561b4092922de8f030bad679c0fcb9a70"}
Jul 03 09:43:46.963438 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.963414    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:46.963524 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.963468    2571 scope.go:117] "RemoveContainer" containerID="f239ab00a953d99e50623b7d3244040692de4733720826831df27809cfc94ae6"
Jul 03 09:43:46.963524 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.963495    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:46.963618 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.963544    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:46.965142 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.965118    2571 generic.go:356] "Generic (PLEG): container finished" podID="785d93fe-f1db-45ab-95b7-4e0d4ccf20fb" containerID="2046a0ccb23d5218e99a29d5341cf7dfca774217f5b33af5cabc2d091f311d5a" exitCode=0
Jul 03 09:43:46.965245 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.965208    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtsgm" event={"ID":"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb","Type":"ContainerDied","Data":"2046a0ccb23d5218e99a29d5341cf7dfca774217f5b33af5cabc2d091f311d5a"}
Jul 03 09:43:46.965425 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.965402    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g8hbm"
Jul 03 09:43:46.966145 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.966127    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g8hbm"
Jul 03 09:43:46.981853 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.981814    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:46.981959 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:46.981936    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:43:47.759200 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:47.758931    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:47.759611 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:47.759325    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:47.971853 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:47.971812    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 09:43:47.972310 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:47.972284    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd" event={"ID":"d915f962-809c-4394-aa7b-72b49fdde9b2","Type":"ContainerStarted","Data":"8b2f44ca8a18a772eaa7db9c92bc55663ad8b755dab8b78ce1c8ddcd67ecc6ce"}
Jul 03 09:43:48.005734 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:48.005684    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd" podStartSLOduration=8.357101887 podStartE2EDuration="29.005669493s" totalImagesPullingTime="20.648567606s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 09:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:43:48.005095949 +0000 UTC m=+29.870384582" watchObservedRunningTime="2026-07-03 09:43:48.005669493 +0000 UTC m=+29.870958148"
Jul 03 09:43:48.411458 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:48.411426    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n4gfv"]
Jul 03 09:43:48.411694 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:48.411611    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:48.411782 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:48.411757    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:48.415488 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:48.415440    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-txhn2"]
Jul 03 09:43:48.415615 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:48.415581    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:48.415707 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:48.415688    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:48.416210 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:48.416187    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d6fcm"]
Jul 03 09:43:48.416322 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:48.416277    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:48.416382 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:48.416366    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:48.976382 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:48.976349    2571 generic.go:356] "Generic (PLEG): container finished" podID="785d93fe-f1db-45ab-95b7-4e0d4ccf20fb" containerID="f787c2c1cfc15809c5af4dc0c9c2ad27cdae8cccddc6a8a43b1d9f489c0425f5" exitCode=0
Jul 03 09:43:48.976855 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:48.976439    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtsgm" event={"ID":"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb","Type":"ContainerDied","Data":"f787c2c1cfc15809c5af4dc0c9c2ad27cdae8cccddc6a8a43b1d9f489c0425f5"}
Jul 03 09:43:49.758640 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:49.758613    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:49.758640 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:49.758626    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:49.758811 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:49.758624    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:49.758811 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:49.758770    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:49.758911 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:49.758891    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:49.758974 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:49.758960    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:50.983419 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:50.983383    2571 generic.go:356] "Generic (PLEG): container finished" podID="785d93fe-f1db-45ab-95b7-4e0d4ccf20fb" containerID="240dfcc9ec0bb762a2822e9799af223fec90bcb183abe4dc12b4988bf99e5a2e" exitCode=0
Jul 03 09:43:50.983789 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:50.983431    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtsgm" event={"ID":"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb","Type":"ContainerDied","Data":"240dfcc9ec0bb762a2822e9799af223fec90bcb183abe4dc12b4988bf99e5a2e"}
Jul 03 09:43:51.332850 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:51.332759    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:51.332850 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:51.332795    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:51.333014 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:51.332903    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:51.333014 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:51.332905    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:51.333014 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:51.332954    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs podName:f18ac135-e44e-413a-9bda-248eacb6287b nodeName:}" failed. No retries permitted until 2026-07-03 09:44:23.33294023 +0000 UTC m=+65.198228862 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs") pod "network-metrics-daemon-txhn2" (UID: "f18ac135-e44e-413a-9bda-248eacb6287b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 09:43:51.333014 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:51.332969    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret podName:d642348a-9842-43db-bcdf-5d84ef42e828 nodeName:}" failed. No retries permitted until 2026-07-03 09:44:23.332960731 +0000 UTC m=+65.198249363 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret") pod "global-pull-secret-syncer-n4gfv" (UID: "d642348a-9842-43db-bcdf-5d84ef42e828") : object "kube-system"/"original-pull-secret" not registered
Jul 03 09:43:51.534242 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:51.534209    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgll7\" (UniqueName: \"kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7\") pod \"network-check-target-d6fcm\" (UID: \"1e886875-0734-4d9e-9ec5-da0aeeefadda\") " pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:51.534429 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:51.534401    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 09:43:51.534429 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:51.534427    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 09:43:51.534537 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:51.534440    2571 projected.go:196] Error preparing data for projected volume kube-api-access-pgll7 for pod openshift-network-diagnostics/network-check-target-d6fcm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:51.534537 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:51.534508    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7 podName:1e886875-0734-4d9e-9ec5-da0aeeefadda nodeName:}" failed. No retries permitted until 2026-07-03 09:44:23.534487535 +0000 UTC m=+65.399776182 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgll7" (UniqueName: "kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7") pod "network-check-target-d6fcm" (UID: "1e886875-0734-4d9e-9ec5-da0aeeefadda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 09:43:51.759038 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:51.759005    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:51.759210 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:51.759005    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:51.759210 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:51.759138    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:51.759210 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:51.759196    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:51.759210 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:51.759012    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:51.759386 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:51.759271    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:52.806304 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:52.806069    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:52.894861 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:52.894802    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:52.996550 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:52.996514    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:53.758657 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:53.758616    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:53.758657 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:53.758638    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:53.758911 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:53.758628    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:53.758911 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:53.758727    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6fcm" podUID="1e886875-0734-4d9e-9ec5-da0aeeefadda"
Jul 03 09:43:53.758911 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:53.758783    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4gfv" podUID="d642348a-9842-43db-bcdf-5d84ef42e828"
Jul 03 09:43:53.758911 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:53.758880    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:43:54.440530 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.440499    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-129.ec2.internal" event="NodeReady"
Jul 03 09:43:54.441005 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.440639    2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 03 09:43:54.490048 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.490020    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wbkqf"]
Jul 03 09:43:54.492612 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.492584    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:54.495250 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.495226    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 03 09:43:54.495520 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.495498    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 03 09:43:54.495928 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.495903    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7ckvl\""
Jul 03 09:43:54.497981 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.497961    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kqx5h"]
Jul 03 09:43:54.499781 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.499761    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kqx5h"
Jul 03 09:43:54.502272 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.502248    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 03 09:43:54.502383 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.502347    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 03 09:43:54.502440 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.502400    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 03 09:43:54.502568 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.502505    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bjlsv\""
Jul 03 09:43:54.503984 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.503923    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wbkqf"]
Jul 03 09:43:54.509472 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.509413    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kqx5h"]
Jul 03 09:43:54.554688 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.554648    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/889943a6-a12d-4476-ba1e-d0fcc7de0f42-tmp-dir\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:54.554873 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.554699    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/889943a6-a12d-4476-ba1e-d0fcc7de0f42-config-volume\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:54.554873 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.554724    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6mlb\" (UniqueName: \"kubernetes.io/projected/889943a6-a12d-4476-ba1e-d0fcc7de0f42-kube-api-access-h6mlb\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:54.554873 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.554790    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:54.655614 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.655574    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/889943a6-a12d-4476-ba1e-d0fcc7de0f42-tmp-dir\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:54.655802 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.655635    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/889943a6-a12d-4476-ba1e-d0fcc7de0f42-config-volume\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:54.655802 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.655661    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6mlb\" (UniqueName: \"kubernetes.io/projected/889943a6-a12d-4476-ba1e-d0fcc7de0f42-kube-api-access-h6mlb\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:54.655802 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.655685    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ph75\" (UniqueName: \"kubernetes.io/projected/5fc5c4fd-aaba-4149-86aa-ded1a82da601-kube-api-access-6ph75\") pod \"ingress-canary-kqx5h\" (UID: \"5fc5c4fd-aaba-4149-86aa-ded1a82da601\") " pod="openshift-ingress-canary/ingress-canary-kqx5h"
Jul 03 09:43:54.655802 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.655717    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:54.655802 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.655793    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fc5c4fd-aaba-4149-86aa-ded1a82da601-cert\") pod \"ingress-canary-kqx5h\" (UID: \"5fc5c4fd-aaba-4149-86aa-ded1a82da601\") " pod="openshift-ingress-canary/ingress-canary-kqx5h"
Jul 03 09:43:54.656080 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:54.655916    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 09:43:54.656080 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:54.655996    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls podName:889943a6-a12d-4476-ba1e-d0fcc7de0f42 nodeName:}" failed. No retries permitted until 2026-07-03 09:43:55.155975344 +0000 UTC m=+37.021263980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls") pod "dns-default-wbkqf" (UID: "889943a6-a12d-4476-ba1e-d0fcc7de0f42") : secret "dns-default-metrics-tls" not found
Jul 03 09:43:54.656080 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.656002    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/889943a6-a12d-4476-ba1e-d0fcc7de0f42-tmp-dir\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:54.656374 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.656354    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/889943a6-a12d-4476-ba1e-d0fcc7de0f42-config-volume\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:54.666846 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.666804    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6mlb\" (UniqueName: \"kubernetes.io/projected/889943a6-a12d-4476-ba1e-d0fcc7de0f42-kube-api-access-h6mlb\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:54.757166 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.757073    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fc5c4fd-aaba-4149-86aa-ded1a82da601-cert\") pod \"ingress-canary-kqx5h\" (UID: \"5fc5c4fd-aaba-4149-86aa-ded1a82da601\") " pod="openshift-ingress-canary/ingress-canary-kqx5h"
Jul 03 09:43:54.757166 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.757163    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ph75\" (UniqueName: \"kubernetes.io/projected/5fc5c4fd-aaba-4149-86aa-ded1a82da601-kube-api-access-6ph75\") pod \"ingress-canary-kqx5h\" (UID: \"5fc5c4fd-aaba-4149-86aa-ded1a82da601\") " pod="openshift-ingress-canary/ingress-canary-kqx5h"
Jul 03 09:43:54.759754 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.759727    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fc5c4fd-aaba-4149-86aa-ded1a82da601-cert\") pod \"ingress-canary-kqx5h\" (UID: \"5fc5c4fd-aaba-4149-86aa-ded1a82da601\") " pod="openshift-ingress-canary/ingress-canary-kqx5h"
Jul 03 09:43:54.766671 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.766638    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ph75\" (UniqueName: \"kubernetes.io/projected/5fc5c4fd-aaba-4149-86aa-ded1a82da601-kube-api-access-6ph75\") pod \"ingress-canary-kqx5h\" (UID: \"5fc5c4fd-aaba-4149-86aa-ded1a82da601\") " pod="openshift-ingress-canary/ingress-canary-kqx5h"
Jul 03 09:43:54.812762 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:54.812723    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kqx5h"
Jul 03 09:43:55.158207 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:55.158023    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kqx5h"]
Jul 03 09:43:55.160157 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:55.160135    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:55.160253 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:55.160238    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 09:43:55.160315 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:55.160290    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls podName:889943a6-a12d-4476-ba1e-d0fcc7de0f42 nodeName:}" failed. No retries permitted until 2026-07-03 09:43:56.160277182 +0000 UTC m=+38.025565814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls") pod "dns-default-wbkqf" (UID: "889943a6-a12d-4476-ba1e-d0fcc7de0f42") : secret "dns-default-metrics-tls" not found
Jul 03 09:43:55.161866 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:43:55.161821    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc5c4fd_aaba_4149_86aa_ded1a82da601.slice/crio-3cacb611659b11f228f5c2b0ae1e5da489f87bd329e51104ba984b224d4b6ddb WatchSource:0}: Error finding container 3cacb611659b11f228f5c2b0ae1e5da489f87bd329e51104ba984b224d4b6ddb: Status 404 returned error can't find the container with id 3cacb611659b11f228f5c2b0ae1e5da489f87bd329e51104ba984b224d4b6ddb
Jul 03 09:43:55.479392 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:55.479365    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:55.580293 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:55.580258    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:55.758302 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:55.758220    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:43:55.758467 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:55.758220    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:43:55.758467 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:55.758209    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:43:55.762004 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:55.761981    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-98cbg\""
Jul 03 09:43:55.762004 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:55.761992    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 03 09:43:55.762201 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:55.762036    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bb7nd\""
Jul 03 09:43:55.762319 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:55.762305    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 03 09:43:55.762375 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:55.762316    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 03 09:43:55.762375 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:55.762337    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 03 09:43:56.001329 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:56.001301    2571 generic.go:356] "Generic (PLEG): container finished" podID="785d93fe-f1db-45ab-95b7-4e0d4ccf20fb" containerID="ac1c6108129fb7f9f513f47310c112e7638e4f728f2753cf3b5d711646331a51" exitCode=0
Jul 03 09:43:56.001487 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:56.001384    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtsgm" event={"ID":"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb","Type":"ContainerDied","Data":"ac1c6108129fb7f9f513f47310c112e7638e4f728f2753cf3b5d711646331a51"}
Jul 03 09:43:56.002891 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:56.002865    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kqx5h" event={"ID":"5fc5c4fd-aaba-4149-86aa-ded1a82da601","Type":"ContainerStarted","Data":"3cacb611659b11f228f5c2b0ae1e5da489f87bd329e51104ba984b224d4b6ddb"}
Jul 03 09:43:56.167106 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:56.167072    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:56.167271 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:56.167249    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 09:43:56.167330 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:56.167321    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls podName:889943a6-a12d-4476-ba1e-d0fcc7de0f42 nodeName:}" failed. No retries permitted until 2026-07-03 09:43:58.167305295 +0000 UTC m=+40.032593935 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls") pod "dns-default-wbkqf" (UID: "889943a6-a12d-4476-ba1e-d0fcc7de0f42") : secret "dns-default-metrics-tls" not found
Jul 03 09:43:57.009103 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:57.009066    2571 generic.go:356] "Generic (PLEG): container finished" podID="785d93fe-f1db-45ab-95b7-4e0d4ccf20fb" containerID="01e41eed5f19457ade9b163b703e51916b8fff1113155cfda920fdc943010d56" exitCode=0
Jul 03 09:43:57.009620 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:57.009148    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtsgm" event={"ID":"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb","Type":"ContainerDied","Data":"01e41eed5f19457ade9b163b703e51916b8fff1113155cfda920fdc943010d56"}
Jul 03 09:43:57.850527 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:57.850360    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:57.951149 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:57.951117    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:58.014333 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:58.014304    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dtsgm" event={"ID":"785d93fe-f1db-45ab-95b7-4e0d4ccf20fb","Type":"ContainerStarted","Data":"e93ecb9b402b05d25772686bf58aa7ee3c8ff92ba995adc2c4d137c8ce270904"}
Jul 03 09:43:58.015747 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:58.015726    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kqx5h" event={"ID":"5fc5c4fd-aaba-4149-86aa-ded1a82da601","Type":"ContainerStarted","Data":"e0aaa712a1bce474d5c1000cfcc182fc3cad74610497ce670bc0bcbd8c56890c"}
Jul 03 09:43:58.039959 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:58.039899    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dtsgm" podStartSLOduration=8.178783044 podStartE2EDuration="39.039880531s" totalImagesPullingTime="30.861097487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 09:43:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:43:58.03473256 +0000 UTC m=+39.900021213" watchObservedRunningTime="2026-07-03 09:43:58.039880531 +0000 UTC m=+39.905169186"
Jul 03 09:43:58.052673 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:58.052575    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kqx5h" podStartSLOduration=1.7086941900000001 podStartE2EDuration="4.052558172s" totalImagesPullingTime="2.343863982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:43:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:43:58.052463771 +0000 UTC m=+39.917752425" watchObservedRunningTime="2026-07-03 09:43:58.052558172 +0000 UTC m=+39.917846826"
Jul 03 09:43:58.131434 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:58.131393    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:58.182763 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:58.182737    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:43:58.182913 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:58.182884    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 09:43:58.183011 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:43:58.182982    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls podName:889943a6-a12d-4476-ba1e-d0fcc7de0f42 nodeName:}" failed. No retries permitted until 2026-07-03 09:44:02.18295739 +0000 UTC m=+44.048246026 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls") pod "dns-default-wbkqf" (UID: "889943a6-a12d-4476-ba1e-d0fcc7de0f42") : secret "dns-default-metrics-tls" not found
Jul 03 09:43:58.185673 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:58.185655    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:58.242407 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:58.242377    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:59.029171 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:59.029133    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:59.081005 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:59.080979    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:59.647185 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:59.647155    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:59.699418 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:59.699392    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:43:59.754876 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:43:59.754847    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:44:00.267198 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:00.267164    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:44:00.367970 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:00.367936    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:44:00.743110 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:00.743081    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:44:00.797716 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:00.797688    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:44:02.211707 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:02.211674    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:44:02.212087 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:02.211819    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 09:44:02.212087 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:02.211892    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls podName:889943a6-a12d-4476-ba1e-d0fcc7de0f42 nodeName:}" failed. No retries permitted until 2026-07-03 09:44:10.211875596 +0000 UTC m=+52.077164232 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls") pod "dns-default-wbkqf" (UID: "889943a6-a12d-4476-ba1e-d0fcc7de0f42") : secret "dns-default-metrics-tls" not found
Jul 03 09:44:10.271545 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:10.271485    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:44:10.272049 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:10.271645    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 09:44:10.272049 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:10.271716    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls podName:889943a6-a12d-4476-ba1e-d0fcc7de0f42 nodeName:}" failed. No retries permitted until 2026-07-03 09:44:26.27170126 +0000 UTC m=+68.136989892 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls") pod "dns-default-wbkqf" (UID: "889943a6-a12d-4476-ba1e-d0fcc7de0f42") : secret "dns-default-metrics-tls" not found
Jul 03 09:44:18.617260 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:18.617228    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:44:18.687965 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:18.687929    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:44:18.998524 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:18.998491    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hmsd"
Jul 03 09:44:19.161734 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.161704    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:44:19.237367 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.237329    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:44:19.454989 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.454956    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-597978765c-8khg5"]
Jul 03 09:44:19.489539 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.489511    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-597978765c-8khg5"]
Jul 03 09:44:19.489682 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.489613    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.492057 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.492034    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 03 09:44:19.492442 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.492427    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7jhcf\""
Jul 03 09:44:19.492496 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.492461    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 03 09:44:19.492496 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.492467    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 03 09:44:19.497652 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.497630    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 03 09:44:19.640105 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.640070    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c15a693-df72-4df6-84ee-0084b4c4de43-ca-trust-extracted\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.640105 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.640111    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj87x\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-kube-api-access-pj87x\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.640556 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.640157    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c15a693-df72-4df6-84ee-0084b4c4de43-image-registry-private-configuration\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.640556 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.640175    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c15a693-df72-4df6-84ee-0084b4c4de43-trusted-ca\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.640556 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.640203    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-certificates\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.640556 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.640218    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-bound-sa-token\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.640556 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.640286    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c15a693-df72-4df6-84ee-0084b4c4de43-installation-pull-secrets\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.640556 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.640313    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.740904 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.740796    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c15a693-df72-4df6-84ee-0084b4c4de43-image-registry-private-configuration\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.740904 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.740860    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c15a693-df72-4df6-84ee-0084b4c4de43-trusted-ca\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.740904 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.740895    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-certificates\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.741161 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.740913    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-bound-sa-token\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.741161 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.740950    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c15a693-df72-4df6-84ee-0084b4c4de43-installation-pull-secrets\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.741161 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.741107    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.741314 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.741173    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c15a693-df72-4df6-84ee-0084b4c4de43-ca-trust-extracted\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.741314 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:19.741220    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 09:44:19.741314 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:19.741243    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597978765c-8khg5: secret "image-registry-tls" not found
Jul 03 09:44:19.741452 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:19.741318    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls podName:0c15a693-df72-4df6-84ee-0084b4c4de43 nodeName:}" failed. No retries permitted until 2026-07-03 09:44:20.241287503 +0000 UTC m=+62.106576158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls") pod "image-registry-597978765c-8khg5" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43") : secret "image-registry-tls" not found
Jul 03 09:44:19.741452 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.741223    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj87x\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-kube-api-access-pj87x\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.741669 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.741649    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-certificates\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.741751 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.741733    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c15a693-df72-4df6-84ee-0084b4c4de43-ca-trust-extracted\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.742179 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.742162    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c15a693-df72-4df6-84ee-0084b4c4de43-trusted-ca\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.744744 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.744721    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c15a693-df72-4df6-84ee-0084b4c4de43-installation-pull-secrets\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.744856 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.744750    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c15a693-df72-4df6-84ee-0084b4c4de43-image-registry-private-configuration\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.750481 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.750461    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-bound-sa-token\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.750577 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.750553    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj87x\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-kube-api-access-pj87x\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:19.966602 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:19.966571    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:44:20.027108 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:20.027024    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:44:20.244697 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:20.244654    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:20.244883 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:20.244805    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 09:44:20.244883 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:20.244823    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597978765c-8khg5: secret "image-registry-tls" not found
Jul 03 09:44:20.244968 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:20.244891    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls podName:0c15a693-df72-4df6-84ee-0084b4c4de43 nodeName:}" failed. No retries permitted until 2026-07-03 09:44:21.244874082 +0000 UTC m=+63.110162725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls") pod "image-registry-597978765c-8khg5" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43") : secret "image-registry-tls" not found
Jul 03 09:44:21.252008 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:21.251974    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:21.252471 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:21.252098    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 09:44:21.252471 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:21.252109    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597978765c-8khg5: secret "image-registry-tls" not found
Jul 03 09:44:21.252471 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:21.252169    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls podName:0c15a693-df72-4df6-84ee-0084b4c4de43 nodeName:}" failed. No retries permitted until 2026-07-03 09:44:23.252154074 +0000 UTC m=+65.117442707 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls") pod "image-registry-597978765c-8khg5" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43") : secret "image-registry-tls" not found
Jul 03 09:44:23.265763 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.265723    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:23.266221 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:23.265921    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 09:44:23.266221 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:23.265936    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597978765c-8khg5: secret "image-registry-tls" not found
Jul 03 09:44:23.266221 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:23.265998    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls podName:0c15a693-df72-4df6-84ee-0084b4c4de43 nodeName:}" failed. No retries permitted until 2026-07-03 09:44:27.265983507 +0000 UTC m=+69.131272139 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls") pod "image-registry-597978765c-8khg5" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43") : secret "image-registry-tls" not found
Jul 03 09:44:23.366674 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.366636    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:44:23.366674 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.366677    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:44:23.369504 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.369480    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 03 09:44:23.369619 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.369538    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 03 09:44:23.377540 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:23.377520    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 03 09:44:23.377638 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:23.377585    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs podName:f18ac135-e44e-413a-9bda-248eacb6287b nodeName:}" failed. No retries permitted until 2026-07-03 09:45:27.377569893 +0000 UTC m=+129.242858530 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs") pod "network-metrics-daemon-txhn2" (UID: "f18ac135-e44e-413a-9bda-248eacb6287b") : secret "metrics-daemon-secret" not found
Jul 03 09:44:23.379697 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.379674    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d642348a-9842-43db-bcdf-5d84ef42e828-original-pull-secret\") pod \"global-pull-secret-syncer-n4gfv\" (UID: \"d642348a-9842-43db-bcdf-5d84ef42e828\") " pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:44:23.383509 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.383488    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4gfv"
Jul 03 09:44:23.515718 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.515690    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n4gfv"]
Jul 03 09:44:23.519108 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:44:23.519085    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd642348a_9842_43db_bcdf_5d84ef42e828.slice/crio-36d93ce79811cfe39c58864b7ea833ea7edd1cb309be2cba6d7f5e1f8b9f1534 WatchSource:0}: Error finding container 36d93ce79811cfe39c58864b7ea833ea7edd1cb309be2cba6d7f5e1f8b9f1534: Status 404 returned error can't find the container with id 36d93ce79811cfe39c58864b7ea833ea7edd1cb309be2cba6d7f5e1f8b9f1534
Jul 03 09:44:23.568895 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.568854    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgll7\" (UniqueName: \"kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7\") pod \"network-check-target-d6fcm\" (UID: \"1e886875-0734-4d9e-9ec5-da0aeeefadda\") " pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:44:23.571572 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.571556    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 03 09:44:23.581627 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.581611    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 03 09:44:23.592781 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.592756    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgll7\" (UniqueName: \"kubernetes.io/projected/1e886875-0734-4d9e-9ec5-da0aeeefadda-kube-api-access-pgll7\") pod \"network-check-target-d6fcm\" (UID: \"1e886875-0734-4d9e-9ec5-da0aeeefadda\") " pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:44:23.680498 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.680470    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bb7nd\""
Jul 03 09:44:23.688644 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.688627    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:44:23.809780 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:23.809696    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d6fcm"]
Jul 03 09:44:23.812720 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:44:23.812693    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e886875_0734_4d9e_9ec5_da0aeeefadda.slice/crio-4793a63675a2096412cb872e169acc01221a2203b95071451b0102d6fc4d63b4 WatchSource:0}: Error finding container 4793a63675a2096412cb872e169acc01221a2203b95071451b0102d6fc4d63b4: Status 404 returned error can't find the container with id 4793a63675a2096412cb872e169acc01221a2203b95071451b0102d6fc4d63b4
Jul 03 09:44:24.081570 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:24.081455    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d6fcm" event={"ID":"1e886875-0734-4d9e-9ec5-da0aeeefadda","Type":"ContainerStarted","Data":"4793a63675a2096412cb872e169acc01221a2203b95071451b0102d6fc4d63b4"}
Jul 03 09:44:24.082683 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:24.082652    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n4gfv" event={"ID":"d642348a-9842-43db-bcdf-5d84ef42e828","Type":"ContainerStarted","Data":"36d93ce79811cfe39c58864b7ea833ea7edd1cb309be2cba6d7f5e1f8b9f1534"}
Jul 03 09:44:26.292337 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:26.292299    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:44:26.292778 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:26.292432    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 09:44:26.292778 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:26.292488    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls podName:889943a6-a12d-4476-ba1e-d0fcc7de0f42 nodeName:}" failed. No retries permitted until 2026-07-03 09:44:58.292475381 +0000 UTC m=+100.157764015 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls") pod "dns-default-wbkqf" (UID: "889943a6-a12d-4476-ba1e-d0fcc7de0f42") : secret "dns-default-metrics-tls" not found
Jul 03 09:44:27.299741 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:27.299706    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:27.300153 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:27.299859    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 09:44:27.300153 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:27.299881    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597978765c-8khg5: secret "image-registry-tls" not found
Jul 03 09:44:27.300153 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:27.299946    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls podName:0c15a693-df72-4df6-84ee-0084b4c4de43 nodeName:}" failed. No retries permitted until 2026-07-03 09:44:35.299926734 +0000 UTC m=+77.165215377 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls") pod "image-registry-597978765c-8khg5" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43") : secret "image-registry-tls" not found
Jul 03 09:44:28.095225 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:28.095182    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d6fcm" event={"ID":"1e886875-0734-4d9e-9ec5-da0aeeefadda","Type":"ContainerStarted","Data":"f72f4ce0ae643a765f60d52258fa89d8f7e1e81c6b8a155b53f7fb10a1d26e2a"}
Jul 03 09:44:28.095402 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:28.095304    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:44:28.115593 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:28.114197    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-d6fcm" podStartSLOduration=65.725400443 podStartE2EDuration="1m9.114175048s" totalImagesPullingTime="3.388774605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:44:28.111949332 +0000 UTC m=+69.977238002" watchObservedRunningTime="2026-07-03 09:44:28.114175048 +0000 UTC m=+69.979463704"
Jul 03 09:44:30.100880 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:30.100845    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n4gfv" event={"ID":"d642348a-9842-43db-bcdf-5d84ef42e828","Type":"ContainerStarted","Data":"a7e0503602c6dfaa1ba2189c283b83aa06e6277591b91fe889d17311511d9e09"}
Jul 03 09:44:30.116784 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:30.116731    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n4gfv" podStartSLOduration=65.611723388 podStartE2EDuration="1m11.116711113s" totalImagesPullingTime="5.504987725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 09:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:44:30.115698369 +0000 UTC m=+71.980987026" watchObservedRunningTime="2026-07-03 09:44:30.116711113 +0000 UTC m=+71.981999769"
Jul 03 09:44:30.923761 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:30.923731    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:44:35.356779 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:35.356727    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:35.357216 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:35.356869    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 09:44:35.357216 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:35.356881    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597978765c-8khg5: secret "image-registry-tls" not found
Jul 03 09:44:35.357216 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:35.356931    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls podName:0c15a693-df72-4df6-84ee-0084b4c4de43 nodeName:}" failed. No retries permitted until 2026-07-03 09:44:51.356917128 +0000 UTC m=+93.222205761 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls") pod "image-registry-597978765c-8khg5" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43") : secret "image-registry-tls" not found
Jul 03 09:44:51.368531 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:51.368471    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:44:51.368965 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:51.368604    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 09:44:51.368965 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:51.368616    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597978765c-8khg5: secret "image-registry-tls" not found
Jul 03 09:44:51.368965 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:51.368666    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls podName:0c15a693-df72-4df6-84ee-0084b4c4de43 nodeName:}" failed. No retries permitted until 2026-07-03 09:45:23.368651584 +0000 UTC m=+125.233940216 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls") pod "image-registry-597978765c-8khg5" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43") : secret "image-registry-tls" not found
Jul 03 09:44:58.316056 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.315998    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:44:58.316479 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:58.316149    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 09:44:58.316479 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:44:58.316221    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls podName:889943a6-a12d-4476-ba1e-d0fcc7de0f42 nodeName:}" failed. No retries permitted until 2026-07-03 09:46:02.316205402 +0000 UTC m=+164.181494035 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls") pod "dns-default-wbkqf" (UID: "889943a6-a12d-4476-ba1e-d0fcc7de0f42") : secret "dns-default-metrics-tls" not found
Jul 03 09:44:58.467615 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.467585    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"]
Jul 03 09:44:58.471668 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.471649    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:44:58.475859 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.475818    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 03 09:44:58.475859 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.475846    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 03 09:44:58.477078 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.477057    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 03 09:44:58.477191 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.477080    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 03 09:44:58.486972 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.486951    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"]
Jul 03 09:44:58.517461 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.517432    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qb2p\" (UniqueName: \"kubernetes.io/projected/dbfe19fb-b6a3-4ce0-a33c-50298b4ec286-kube-api-access-7qb2p\") pod \"klusterlet-addon-workmgr-579dc79546-tglm9\" (UID: \"dbfe19fb-b6a3-4ce0-a33c-50298b4ec286\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:44:58.517595 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.517488    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbfe19fb-b6a3-4ce0-a33c-50298b4ec286-tmp\") pod \"klusterlet-addon-workmgr-579dc79546-tglm9\" (UID: \"dbfe19fb-b6a3-4ce0-a33c-50298b4ec286\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:44:58.517595 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.517521    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/dbfe19fb-b6a3-4ce0-a33c-50298b4ec286-klusterlet-config\") pod \"klusterlet-addon-workmgr-579dc79546-tglm9\" (UID: \"dbfe19fb-b6a3-4ce0-a33c-50298b4ec286\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:44:58.618161 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.618068    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/dbfe19fb-b6a3-4ce0-a33c-50298b4ec286-klusterlet-config\") pod \"klusterlet-addon-workmgr-579dc79546-tglm9\" (UID: \"dbfe19fb-b6a3-4ce0-a33c-50298b4ec286\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:44:58.618161 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.618119    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qb2p\" (UniqueName: \"kubernetes.io/projected/dbfe19fb-b6a3-4ce0-a33c-50298b4ec286-kube-api-access-7qb2p\") pod \"klusterlet-addon-workmgr-579dc79546-tglm9\" (UID: \"dbfe19fb-b6a3-4ce0-a33c-50298b4ec286\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:44:58.618161 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.618157    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbfe19fb-b6a3-4ce0-a33c-50298b4ec286-tmp\") pod \"klusterlet-addon-workmgr-579dc79546-tglm9\" (UID: \"dbfe19fb-b6a3-4ce0-a33c-50298b4ec286\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:44:58.618480 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.618463    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbfe19fb-b6a3-4ce0-a33c-50298b4ec286-tmp\") pod \"klusterlet-addon-workmgr-579dc79546-tglm9\" (UID: \"dbfe19fb-b6a3-4ce0-a33c-50298b4ec286\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:44:58.620515 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.620497    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/dbfe19fb-b6a3-4ce0-a33c-50298b4ec286-klusterlet-config\") pod \"klusterlet-addon-workmgr-579dc79546-tglm9\" (UID: \"dbfe19fb-b6a3-4ce0-a33c-50298b4ec286\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:44:58.626603 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.626577    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qb2p\" (UniqueName: \"kubernetes.io/projected/dbfe19fb-b6a3-4ce0-a33c-50298b4ec286-kube-api-access-7qb2p\") pod \"klusterlet-addon-workmgr-579dc79546-tglm9\" (UID: \"dbfe19fb-b6a3-4ce0-a33c-50298b4ec286\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:44:58.780568 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.780534    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:44:58.907312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:58.907284    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"]
Jul 03 09:44:58.910171 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:44:58.910128    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbfe19fb_b6a3_4ce0_a33c_50298b4ec286.slice/crio-d040de30a99d78614162b84c4504244d2df80a0518739250fa60e0a0938c999f WatchSource:0}: Error finding container d040de30a99d78614162b84c4504244d2df80a0518739250fa60e0a0938c999f: Status 404 returned error can't find the container with id d040de30a99d78614162b84c4504244d2df80a0518739250fa60e0a0938c999f
Jul 03 09:44:59.100061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:59.100033    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-d6fcm"
Jul 03 09:44:59.175544 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:44:59.175466    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9" event={"ID":"dbfe19fb-b6a3-4ce0-a33c-50298b4ec286","Type":"ContainerStarted","Data":"d040de30a99d78614162b84c4504244d2df80a0518739250fa60e0a0938c999f"}
Jul 03 09:45:03.187018 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:45:03.186981    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9" event={"ID":"dbfe19fb-b6a3-4ce0-a33c-50298b4ec286","Type":"ContainerStarted","Data":"e0d1f4b7e7c59b635b6ef425a37b02cec1ed3257b7d5b03a7e4cdbed82f8d377"}
Jul 03 09:45:03.187440 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:45:03.187128    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:45:03.188733 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:45:03.188707    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:45:03.206636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:45:03.206586    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9" podStartSLOduration=1.546618129 podStartE2EDuration="5.206565428s" totalImagesPullingTime="3.659947299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:45:03.205871818 +0000 UTC m=+105.071160466" watchObservedRunningTime="2026-07-03 09:45:03.206565428 +0000 UTC m=+105.071854085"
Jul 03 09:45:12.469158 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:45:12.469121    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:45:12.526375 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:45:12.526345    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:45:23.402409 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:45:23.402352    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:45:23.402938 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:45:23.402522    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 09:45:23.402938 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:45:23.402536    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597978765c-8khg5: secret "image-registry-tls" not found
Jul 03 09:45:23.402938 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:45:23.402593    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls podName:0c15a693-df72-4df6-84ee-0084b4c4de43 nodeName:}" failed. No retries permitted until 2026-07-03 09:46:27.402576346 +0000 UTC m=+189.267864996 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls") pod "image-registry-597978765c-8khg5" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43") : secret "image-registry-tls" not found
Jul 03 09:45:27.430858 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:45:27.430783    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:45:27.431277 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:45:27.430929    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 03 09:45:27.431277 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:45:27.430990    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs podName:f18ac135-e44e-413a-9bda-248eacb6287b nodeName:}" failed. No retries permitted until 2026-07-03 09:47:29.430973889 +0000 UTC m=+251.296262522 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs") pod "network-metrics-daemon-txhn2" (UID: "f18ac135-e44e-413a-9bda-248eacb6287b") : secret "metrics-daemon-secret" not found
Jul 03 09:45:57.505471 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:45:57.505428    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-wbkqf" podUID="889943a6-a12d-4476-ba1e-d0fcc7de0f42"
Jul 03 09:45:58.317711 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:45:58.317680    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:45:58.770405 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:45:58.770369    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:46:02.373190 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:46:02.373139    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:46:02.373683 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:46:02.373309    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 09:46:02.373683 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:46:02.373401    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls podName:889943a6-a12d-4476-ba1e-d0fcc7de0f42 nodeName:}" failed. No retries permitted until 2026-07-03 09:48:04.373381189 +0000 UTC m=+286.238669822 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls") pod "dns-default-wbkqf" (UID: "889943a6-a12d-4476-ba1e-d0fcc7de0f42") : secret "dns-default-metrics-tls" not found
Jul 03 09:46:11.758217 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:46:11.758122    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:46:22.499749 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:46:22.499703    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-597978765c-8khg5" podUID="0c15a693-df72-4df6-84ee-0084b4c4de43"
Jul 03 09:46:27.444663 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:46:27.444609    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls\") pod \"image-registry-597978765c-8khg5\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") " pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:46:27.445193 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:46:27.444818    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 09:46:27.445193 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:46:27.444865    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-597978765c-8khg5: secret "image-registry-tls" not found
Jul 03 09:46:27.445193 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:46:27.444938    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls podName:0c15a693-df72-4df6-84ee-0084b4c4de43 nodeName:}" failed. No retries permitted until 2026-07-03 09:48:29.444915782 +0000 UTC m=+311.310204419 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls") pod "image-registry-597978765c-8khg5" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43") : secret "image-registry-tls" not found
Jul 03 09:46:34.758399 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:46:34.758360    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:47:03.187596 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:03.187538    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9" podUID="dbfe19fb-b6a3-4ce0-a33c-50298b4ec286" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused"
Jul 03 09:47:03.470789 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:03.470709    2571 generic.go:356] "Generic (PLEG): container finished" podID="dbfe19fb-b6a3-4ce0-a33c-50298b4ec286" containerID="e0d1f4b7e7c59b635b6ef425a37b02cec1ed3257b7d5b03a7e4cdbed82f8d377" exitCode=1
Jul 03 09:47:03.470789 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:03.470745    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9" event={"ID":"dbfe19fb-b6a3-4ce0-a33c-50298b4ec286","Type":"ContainerDied","Data":"e0d1f4b7e7c59b635b6ef425a37b02cec1ed3257b7d5b03a7e4cdbed82f8d377"}
Jul 03 09:47:03.471076 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:03.471061    2571 scope.go:117] "RemoveContainer" containerID="e0d1f4b7e7c59b635b6ef425a37b02cec1ed3257b7d5b03a7e4cdbed82f8d377"
Jul 03 09:47:04.477236 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:04.477195    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9" event={"ID":"dbfe19fb-b6a3-4ce0-a33c-50298b4ec286","Type":"ContainerStarted","Data":"c18829d1cb026d5dfec08825c90f90b8ad9683a6023a5bc8cb2f1b3251d90917"}
Jul 03 09:47:04.477651 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:04.477518    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:47:04.478066 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:04.478047    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-579dc79546-tglm9"
Jul 03 09:47:09.061285 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:09.061255    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:29.481148 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:29.481105    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:47:29.481606 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:29.481252    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 03 09:47:29.481606 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:29.481325    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs podName:f18ac135-e44e-413a-9bda-248eacb6287b nodeName:}" failed. No retries permitted until 2026-07-03 09:49:31.481305115 +0000 UTC m=+373.346593755 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs") pod "network-metrics-daemon-txhn2" (UID: "f18ac135-e44e-413a-9bda-248eacb6287b") : secret "metrics-daemon-secret" not found
Jul 03 09:47:36.441280 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.441245    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-shl4s"]
Jul 03 09:47:36.444078 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.444060    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-shl4s"
Jul 03 09:47:36.445656 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.445629    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"]
Jul 03 09:47:36.446578 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.446558    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 03 09:47:36.447503 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.447484    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 03 09:47:36.447587 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.447567    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-594s8\""
Jul 03 09:47:36.448446 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.448428    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"
Jul 03 09:47:36.452509 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.452493    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 03 09:47:36.452590 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.452540    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 03 09:47:36.453122 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.453104    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 03 09:47:36.453593 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.453578    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-nsdjs\""
Jul 03 09:47:36.453985 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.453967    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 03 09:47:36.460104 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.460083    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-shl4s"]
Jul 03 09:47:36.463262 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.463237    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"]
Jul 03 09:47:36.534673 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.534639    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mvqt\" (UniqueName: \"kubernetes.io/projected/901fcf7f-0f02-41d9-a471-55471d7ecbe7-kube-api-access-6mvqt\") pod \"volume-data-source-validator-5d456fdbcd-shl4s\" (UID: \"901fcf7f-0f02-41d9-a471-55471d7ecbe7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-shl4s"
Jul 03 09:47:36.534673 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.534683    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7bcdf3e0-7d30-4010-9219-8a8901e43f25-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wpk2r\" (UID: \"7bcdf3e0-7d30-4010-9219-8a8901e43f25\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"
Jul 03 09:47:36.534924 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.534701    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj4z9\" (UniqueName: \"kubernetes.io/projected/7bcdf3e0-7d30-4010-9219-8a8901e43f25-kube-api-access-wj4z9\") pod \"kube-storage-version-migrator-operator-846bf668f7-wpk2r\" (UID: \"7bcdf3e0-7d30-4010-9219-8a8901e43f25\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"
Jul 03 09:47:36.534924 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.534753    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7bcdf3e0-7d30-4010-9219-8a8901e43f25-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wpk2r\" (UID: \"7bcdf3e0-7d30-4010-9219-8a8901e43f25\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"
Jul 03 09:47:36.541737 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.541710    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-c5cd9c95-m6fvt"]
Jul 03 09:47:36.544650 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.544633    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.546944 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.546922    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 03 09:47:36.547074 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.546950    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vfnhs\""
Jul 03 09:47:36.547074 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.547019    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 03 09:47:36.547247 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.547227    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 03 09:47:36.547316 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.547289    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 03 09:47:36.547363 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.547312    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 03 09:47:36.547414 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.547398    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 03 09:47:36.559907 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.559884    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c5cd9c95-m6fvt"]
Jul 03 09:47:36.636077 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.636036    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-default-certificate\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.636077 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.636075    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.636333 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.636105    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-stats-auth\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.636333 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.636204    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mvqt\" (UniqueName: \"kubernetes.io/projected/901fcf7f-0f02-41d9-a471-55471d7ecbe7-kube-api-access-6mvqt\") pod \"volume-data-source-validator-5d456fdbcd-shl4s\" (UID: \"901fcf7f-0f02-41d9-a471-55471d7ecbe7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-shl4s"
Jul 03 09:47:36.636333 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.636233    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2msn\" (UniqueName: \"kubernetes.io/projected/865a12bb-5687-4bc6-8f85-8a5df5db37fa-kube-api-access-w2msn\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.636333 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.636257    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7bcdf3e0-7d30-4010-9219-8a8901e43f25-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wpk2r\" (UID: \"7bcdf3e0-7d30-4010-9219-8a8901e43f25\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"
Jul 03 09:47:36.636333 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.636282    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj4z9\" (UniqueName: \"kubernetes.io/projected/7bcdf3e0-7d30-4010-9219-8a8901e43f25-kube-api-access-wj4z9\") pod \"kube-storage-version-migrator-operator-846bf668f7-wpk2r\" (UID: \"7bcdf3e0-7d30-4010-9219-8a8901e43f25\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"
Jul 03 09:47:36.636333 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.636312    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7bcdf3e0-7d30-4010-9219-8a8901e43f25-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wpk2r\" (UID: \"7bcdf3e0-7d30-4010-9219-8a8901e43f25\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"
Jul 03 09:47:36.636554 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.636338    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.636809 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.636779    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7bcdf3e0-7d30-4010-9219-8a8901e43f25-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wpk2r\" (UID: \"7bcdf3e0-7d30-4010-9219-8a8901e43f25\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"
Jul 03 09:47:36.638538 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.638517    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7bcdf3e0-7d30-4010-9219-8a8901e43f25-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wpk2r\" (UID: \"7bcdf3e0-7d30-4010-9219-8a8901e43f25\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"
Jul 03 09:47:36.644102 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.644078    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj4z9\" (UniqueName: \"kubernetes.io/projected/7bcdf3e0-7d30-4010-9219-8a8901e43f25-kube-api-access-wj4z9\") pod \"kube-storage-version-migrator-operator-846bf668f7-wpk2r\" (UID: \"7bcdf3e0-7d30-4010-9219-8a8901e43f25\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"
Jul 03 09:47:36.644453 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.644431    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mvqt\" (UniqueName: \"kubernetes.io/projected/901fcf7f-0f02-41d9-a471-55471d7ecbe7-kube-api-access-6mvqt\") pod \"volume-data-source-validator-5d456fdbcd-shl4s\" (UID: \"901fcf7f-0f02-41d9-a471-55471d7ecbe7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-shl4s"
Jul 03 09:47:36.687263 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.687234    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:36.737405 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.737322    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2msn\" (UniqueName: \"kubernetes.io/projected/865a12bb-5687-4bc6-8f85-8a5df5db37fa-kube-api-access-w2msn\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.737405 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.737364    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.737405 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.737396    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-default-certificate\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.737626 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.737417    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.737626 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.737446    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-stats-auth\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.737626 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:36.737514    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle podName:865a12bb-5687-4bc6-8f85-8a5df5db37fa nodeName:}" failed. No retries permitted until 2026-07-03 09:47:37.237493418 +0000 UTC m=+259.102782081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle") pod "router-default-c5cd9c95-m6fvt" (UID: "865a12bb-5687-4bc6-8f85-8a5df5db37fa") : configmap references non-existent config key: service-ca.crt
Jul 03 09:47:36.737626 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:36.737531    2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 03 09:47:36.737626 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:36.737599    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs podName:865a12bb-5687-4bc6-8f85-8a5df5db37fa nodeName:}" failed. No retries permitted until 2026-07-03 09:47:37.237580532 +0000 UTC m=+259.102869176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs") pod "router-default-c5cd9c95-m6fvt" (UID: "865a12bb-5687-4bc6-8f85-8a5df5db37fa") : secret "router-metrics-certs-default" not found
Jul 03 09:47:36.740073 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.740047    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-default-certificate\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.740193 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.740050    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-stats-auth\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.747843 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.747809    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2msn\" (UniqueName: \"kubernetes.io/projected/865a12bb-5687-4bc6-8f85-8a5df5db37fa-kube-api-access-w2msn\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:36.755184 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.755165    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-shl4s"
Jul 03 09:47:36.759988 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.759965    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"
Jul 03 09:47:36.793596 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.793567    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:36.889567 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.889533    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r"]
Jul 03 09:47:36.894227 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:47:36.894197    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bcdf3e0_7d30_4010_9219_8a8901e43f25.slice/crio-4b80809af0e63588407cc68709f81ae27f000bb76e6f2f3d26a71da4f8ebc17d WatchSource:0}: Error finding container 4b80809af0e63588407cc68709f81ae27f000bb76e6f2f3d26a71da4f8ebc17d: Status 404 returned error can't find the container with id 4b80809af0e63588407cc68709f81ae27f000bb76e6f2f3d26a71da4f8ebc17d
Jul 03 09:47:36.904808 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:36.904782    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-shl4s"]
Jul 03 09:47:36.907660 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:47:36.907638    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod901fcf7f_0f02_41d9_a471_55471d7ecbe7.slice/crio-b751e00eb050bd3c0e37d3e8645453de3399279fe4a49c3a9ec97581c6cf7a41 WatchSource:0}: Error finding container b751e00eb050bd3c0e37d3e8645453de3399279fe4a49c3a9ec97581c6cf7a41: Status 404 returned error can't find the container with id b751e00eb050bd3c0e37d3e8645453de3399279fe4a49c3a9ec97581c6cf7a41
Jul 03 09:47:37.241582 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:37.241546    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:37.241727 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:37.241598    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:37.241727 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:37.241689    2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 03 09:47:37.241727 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:37.241722    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle podName:865a12bb-5687-4bc6-8f85-8a5df5db37fa nodeName:}" failed. No retries permitted until 2026-07-03 09:47:38.241700461 +0000 UTC m=+260.106989099 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle") pod "router-default-c5cd9c95-m6fvt" (UID: "865a12bb-5687-4bc6-8f85-8a5df5db37fa") : configmap references non-existent config key: service-ca.crt
Jul 03 09:47:37.241872 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:37.241748    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs podName:865a12bb-5687-4bc6-8f85-8a5df5db37fa nodeName:}" failed. No retries permitted until 2026-07-03 09:47:38.241740375 +0000 UTC m=+260.107029007 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs") pod "router-default-c5cd9c95-m6fvt" (UID: "865a12bb-5687-4bc6-8f85-8a5df5db37fa") : secret "router-metrics-certs-default" not found
Jul 03 09:47:37.572797 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:37.572713    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r" event={"ID":"7bcdf3e0-7d30-4010-9219-8a8901e43f25","Type":"ContainerStarted","Data":"4b80809af0e63588407cc68709f81ae27f000bb76e6f2f3d26a71da4f8ebc17d"}
Jul 03 09:47:37.574073 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:37.574044    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-shl4s" event={"ID":"901fcf7f-0f02-41d9-a471-55471d7ecbe7","Type":"ContainerStarted","Data":"b751e00eb050bd3c0e37d3e8645453de3399279fe4a49c3a9ec97581c6cf7a41"}
Jul 03 09:47:38.249320 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:38.249234    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:38.249320 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:38.249319    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:38.249498 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:38.249397    2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 03 09:47:38.249498 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:38.249441    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle podName:865a12bb-5687-4bc6-8f85-8a5df5db37fa nodeName:}" failed. No retries permitted until 2026-07-03 09:47:40.249428127 +0000 UTC m=+262.114716760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle") pod "router-default-c5cd9c95-m6fvt" (UID: "865a12bb-5687-4bc6-8f85-8a5df5db37fa") : configmap references non-existent config key: service-ca.crt
Jul 03 09:47:38.249498 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:38.249457    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs podName:865a12bb-5687-4bc6-8f85-8a5df5db37fa nodeName:}" failed. No retries permitted until 2026-07-03 09:47:40.249450455 +0000 UTC m=+262.114739088 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs") pod "router-default-c5cd9c95-m6fvt" (UID: "865a12bb-5687-4bc6-8f85-8a5df5db37fa") : secret "router-metrics-certs-default" not found
Jul 03 09:47:38.579488 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:38.579403    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-shl4s" event={"ID":"901fcf7f-0f02-41d9-a471-55471d7ecbe7","Type":"ContainerStarted","Data":"818a3e7d9a4f6b6180c759250b2ff02aadd3cb7fa6f8e10d28df143e26d6cd17"}
Jul 03 09:47:38.596132 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:38.596055    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-shl4s" podStartSLOduration=1.530868994 podStartE2EDuration="2.596037204s" totalImagesPullingTime="1.06516821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:47:38.595811597 +0000 UTC m=+260.461100303" watchObservedRunningTime="2026-07-03 09:47:38.596037204 +0000 UTC m=+260.461325889"
Jul 03 09:47:38.706546 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:38.706514    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:38.807506 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:38.807472    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:39.583907 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:39.583869    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r" event={"ID":"7bcdf3e0-7d30-4010-9219-8a8901e43f25","Type":"ContainerStarted","Data":"49516b387ce0b06fcba304e4224a8803dc13b08f353a28604f6b1e4bbead57b5"}
Jul 03 09:47:39.601517 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:39.601466    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r" podStartSLOduration=1.775559538 podStartE2EDuration="3.601450531s" totalImagesPullingTime="1.825890993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:47:39.600583128 +0000 UTC m=+261.465871783" watchObservedRunningTime="2026-07-03 09:47:39.601450531 +0000 UTC m=+261.466739186"
Jul 03 09:47:40.267676 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:40.267633    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:40.267957 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:40.267718    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:40.267957 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:40.267792    2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 03 09:47:40.267957 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:40.267863    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle podName:865a12bb-5687-4bc6-8f85-8a5df5db37fa nodeName:}" failed. No retries permitted until 2026-07-03 09:47:44.267847535 +0000 UTC m=+266.133136181 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle") pod "router-default-c5cd9c95-m6fvt" (UID: "865a12bb-5687-4bc6-8f85-8a5df5db37fa") : configmap references non-existent config key: service-ca.crt
Jul 03 09:47:40.267957 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:40.267880    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs podName:865a12bb-5687-4bc6-8f85-8a5df5db37fa nodeName:}" failed. No retries permitted until 2026-07-03 09:47:44.267874007 +0000 UTC m=+266.133162640 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs") pod "router-default-c5cd9c95-m6fvt" (UID: "865a12bb-5687-4bc6-8f85-8a5df5db37fa") : secret "router-metrics-certs-default" not found
Jul 03 09:47:40.381178 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:40.381105    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:40.460461 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:40.460407    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:40.523525 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:40.523447    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:41.318762 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:41.318730    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:41.387501 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:41.387471    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:41.743123 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:41.743092    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:41.817140 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:41.817099    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:42.025808 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.025712    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:42.130060 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.130017    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:42.332025 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.331953    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gz8ww"]
Jul 03 09:47:42.334915 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.334897    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.337705 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.337457    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 03 09:47:42.337705 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.337543    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 03 09:47:42.340098 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.340076    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 03 09:47:42.340234 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.340115    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c78n8\""
Jul 03 09:47:42.340303 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.340172    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 03 09:47:42.348383 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.348362    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gz8ww"]
Jul 03 09:47:42.373992 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.373963    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:42.439426 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.439390    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:42.486967 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.486928    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.487128 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.486990    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft5c5\" (UniqueName: \"kubernetes.io/projected/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-api-access-ft5c5\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.487128 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.487098    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-crio-socket\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.487222 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.487204    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.487325 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.487300    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-data-volume\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.537676 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.537646    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:42.588595 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.588517    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.588595 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.588564    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-data-volume\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.588595 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.588587    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.588801 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.588623    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft5c5\" (UniqueName: \"kubernetes.io/projected/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-api-access-ft5c5\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.588801 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.588647    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-crio-socket\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.588801 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:42.588650    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 09:47:42.588801 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:42.588725    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm podName:7f1108aa-d8c6-4608-aa94-a85740bfdc1e nodeName:}" failed. No retries permitted until 2026-07-03 09:47:43.088702446 +0000 UTC m=+264.953991087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gz8ww" (UID: "7f1108aa-d8c6-4608-aa94-a85740bfdc1e") : configmap "kube-rbac-proxy" not found
Jul 03 09:47:42.588801 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.588741    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-crio-socket\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.588801 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:42.588773    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 09:47:42.589111 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:42.588823    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls podName:7f1108aa-d8c6-4608-aa94-a85740bfdc1e nodeName:}" failed. No retries permitted until 2026-07-03 09:47:43.088810789 +0000 UTC m=+264.954099424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gz8ww" (UID: "7f1108aa-d8c6-4608-aa94-a85740bfdc1e") : secret "insights-runtime-extractor-tls" not found
Jul 03 09:47:42.589111 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.588946    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-data-volume\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.597463 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.597436    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft5c5\" (UniqueName: \"kubernetes.io/projected/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-api-access-ft5c5\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:42.637985 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:42.637953    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:43.094231 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.094192    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:43.094379 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.094253    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:43.094379 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:43.094314    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 09:47:43.094469 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:43.094383    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 09:47:43.094469 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:43.094387    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm podName:7f1108aa-d8c6-4608-aa94-a85740bfdc1e nodeName:}" failed. No retries permitted until 2026-07-03 09:47:44.094369876 +0000 UTC m=+265.959658509 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gz8ww" (UID: "7f1108aa-d8c6-4608-aa94-a85740bfdc1e") : configmap "kube-rbac-proxy" not found
Jul 03 09:47:43.094469 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:43.094420    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls podName:7f1108aa-d8c6-4608-aa94-a85740bfdc1e nodeName:}" failed. No retries permitted until 2026-07-03 09:47:44.094409293 +0000 UTC m=+265.959697927 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gz8ww" (UID: "7f1108aa-d8c6-4608-aa94-a85740bfdc1e") : secret "insights-runtime-extractor-tls" not found
Jul 03 09:47:43.617393 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.617362    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-hggcv"]
Jul 03 09:47:43.620377 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.620360    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-hggcv"
Jul 03 09:47:43.622811 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.622778    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 03 09:47:43.622944 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.622904    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 03 09:47:43.623076 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.623060    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 03 09:47:43.623942 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.623927    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 03 09:47:43.624032 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.624005    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-b24nj\""
Jul 03 09:47:43.628068 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.628048    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-hggcv"]
Jul 03 09:47:43.656235 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.656197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:43.799383 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.799348    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cd725289-33c4-4e45-a9c9-61c7bb30c9e9-signing-cabundle\") pod \"service-ca-758f599c5-hggcv\" (UID: \"cd725289-33c4-4e45-a9c9-61c7bb30c9e9\") " pod="openshift-service-ca/service-ca-758f599c5-hggcv"
Jul 03 09:47:43.799549 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.799452    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mnq6\" (UniqueName: \"kubernetes.io/projected/cd725289-33c4-4e45-a9c9-61c7bb30c9e9-kube-api-access-8mnq6\") pod \"service-ca-758f599c5-hggcv\" (UID: \"cd725289-33c4-4e45-a9c9-61c7bb30c9e9\") " pod="openshift-service-ca/service-ca-758f599c5-hggcv"
Jul 03 09:47:43.799549 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.799472    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cd725289-33c4-4e45-a9c9-61c7bb30c9e9-signing-key\") pod \"service-ca-758f599c5-hggcv\" (UID: \"cd725289-33c4-4e45-a9c9-61c7bb30c9e9\") " pod="openshift-service-ca/service-ca-758f599c5-hggcv"
Jul 03 09:47:43.900652 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.900623    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cd725289-33c4-4e45-a9c9-61c7bb30c9e9-signing-cabundle\") pod \"service-ca-758f599c5-hggcv\" (UID: \"cd725289-33c4-4e45-a9c9-61c7bb30c9e9\") " pod="openshift-service-ca/service-ca-758f599c5-hggcv"
Jul 03 09:47:43.900780 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.900687    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mnq6\" (UniqueName: \"kubernetes.io/projected/cd725289-33c4-4e45-a9c9-61c7bb30c9e9-kube-api-access-8mnq6\") pod \"service-ca-758f599c5-hggcv\" (UID: \"cd725289-33c4-4e45-a9c9-61c7bb30c9e9\") " pod="openshift-service-ca/service-ca-758f599c5-hggcv"
Jul 03 09:47:43.900780 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.900711    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cd725289-33c4-4e45-a9c9-61c7bb30c9e9-signing-key\") pod \"service-ca-758f599c5-hggcv\" (UID: \"cd725289-33c4-4e45-a9c9-61c7bb30c9e9\") " pod="openshift-service-ca/service-ca-758f599c5-hggcv"
Jul 03 09:47:43.901334 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.901304    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cd725289-33c4-4e45-a9c9-61c7bb30c9e9-signing-cabundle\") pod \"service-ca-758f599c5-hggcv\" (UID: \"cd725289-33c4-4e45-a9c9-61c7bb30c9e9\") " pod="openshift-service-ca/service-ca-758f599c5-hggcv"
Jul 03 09:47:43.903116 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.903094    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cd725289-33c4-4e45-a9c9-61c7bb30c9e9-signing-key\") pod \"service-ca-758f599c5-hggcv\" (UID: \"cd725289-33c4-4e45-a9c9-61c7bb30c9e9\") " pod="openshift-service-ca/service-ca-758f599c5-hggcv"
Jul 03 09:47:43.910008 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.909987    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mnq6\" (UniqueName: \"kubernetes.io/projected/cd725289-33c4-4e45-a9c9-61c7bb30c9e9-kube-api-access-8mnq6\") pod \"service-ca-758f599c5-hggcv\" (UID: \"cd725289-33c4-4e45-a9c9-61c7bb30c9e9\") " pod="openshift-service-ca/service-ca-758f599c5-hggcv"
Jul 03 09:47:43.928943 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:43.928910    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-hggcv"
Jul 03 09:47:44.054366 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:44.054331    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-hggcv"]
Jul 03 09:47:44.057227 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:47:44.057198    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd725289_33c4_4e45_a9c9_61c7bb30c9e9.slice/crio-2d20a444925ead67279cc4d82ccade2766c46973c6562087b8a51349daa82fa6 WatchSource:0}: Error finding container 2d20a444925ead67279cc4d82ccade2766c46973c6562087b8a51349daa82fa6: Status 404 returned error can't find the container with id 2d20a444925ead67279cc4d82ccade2766c46973c6562087b8a51349daa82fa6
Jul 03 09:47:44.102362 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:44.102323    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:44.102503 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:44.102441    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:44.102503 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:44.102466    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 09:47:44.102580 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:44.102511    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 09:47:44.102580 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:44.102527    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls podName:7f1108aa-d8c6-4608-aa94-a85740bfdc1e nodeName:}" failed. No retries permitted until 2026-07-03 09:47:46.102510155 +0000 UTC m=+267.967798790 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gz8ww" (UID: "7f1108aa-d8c6-4608-aa94-a85740bfdc1e") : secret "insights-runtime-extractor-tls" not found
Jul 03 09:47:44.102580 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:44.102554    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm podName:7f1108aa-d8c6-4608-aa94-a85740bfdc1e nodeName:}" failed. No retries permitted until 2026-07-03 09:47:46.102540862 +0000 UTC m=+267.967829495 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gz8ww" (UID: "7f1108aa-d8c6-4608-aa94-a85740bfdc1e") : configmap "kube-rbac-proxy" not found
Jul 03 09:47:44.303906 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:44.303795    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:44.303906 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:44.303900    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:44.304079 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:44.303964    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle podName:865a12bb-5687-4bc6-8f85-8a5df5db37fa nodeName:}" failed. No retries permitted until 2026-07-03 09:47:52.303947041 +0000 UTC m=+274.169235674 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle") pod "router-default-c5cd9c95-m6fvt" (UID: "865a12bb-5687-4bc6-8f85-8a5df5db37fa") : configmap references non-existent config key: service-ca.crt
Jul 03 09:47:44.304079 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:44.304044    2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 03 09:47:44.304151 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:44.304097    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs podName:865a12bb-5687-4bc6-8f85-8a5df5db37fa nodeName:}" failed. No retries permitted until 2026-07-03 09:47:52.304085204 +0000 UTC m=+274.169373836 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs") pod "router-default-c5cd9c95-m6fvt" (UID: "865a12bb-5687-4bc6-8f85-8a5df5db37fa") : secret "router-metrics-certs-default" not found
Jul 03 09:47:44.600757 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:44.600665    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-hggcv" event={"ID":"cd725289-33c4-4e45-a9c9-61c7bb30c9e9","Type":"ContainerStarted","Data":"2d20a444925ead67279cc4d82ccade2766c46973c6562087b8a51349daa82fa6"}
Jul 03 09:47:44.613438 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:44.613403    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:44.915778 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:44.915751    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gmjss_20bc336f-ff0c-4072-8275-46857fb9f016/dns-node-resolver/0.log"
Jul 03 09:47:45.914713 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:45.914678    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fng7l_dd3cac63-b412-453b-90bc-5b7897a9920f/node-ca/0.log"
Jul 03 09:47:46.120066 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:46.119977    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:46.120493 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:46.120067    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:46.120493 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:46.120085    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 09:47:46.120493 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:46.120140    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm podName:7f1108aa-d8c6-4608-aa94-a85740bfdc1e nodeName:}" failed. No retries permitted until 2026-07-03 09:47:50.120125586 +0000 UTC m=+271.985414220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gz8ww" (UID: "7f1108aa-d8c6-4608-aa94-a85740bfdc1e") : configmap "kube-rbac-proxy" not found
Jul 03 09:47:46.120493 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:46.120204    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 09:47:46.120493 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:46.120254    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls podName:7f1108aa-d8c6-4608-aa94-a85740bfdc1e nodeName:}" failed. No retries permitted until 2026-07-03 09:47:50.12024242 +0000 UTC m=+271.985531054 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gz8ww" (UID: "7f1108aa-d8c6-4608-aa94-a85740bfdc1e") : secret "insights-runtime-extractor-tls" not found
Jul 03 09:47:46.608128 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:46.608087    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-hggcv" event={"ID":"cd725289-33c4-4e45-a9c9-61c7bb30c9e9","Type":"ContainerStarted","Data":"0fdaca24cd28859fc3805277064823db4fa04898cc396fab42a8a08b95581bab"}
Jul 03 09:47:46.623652 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:46.623593    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-hggcv" podStartSLOduration=1.834642257 podStartE2EDuration="3.623575997s" totalImagesPullingTime="1.78893374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:47:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:47:46.62278965 +0000 UTC m=+268.488078306" watchObservedRunningTime="2026-07-03 09:47:46.623575997 +0000 UTC m=+268.488864651"
Jul 03 09:47:46.881841 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:46.881742    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:47.120124 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:47.120091    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kqx5h_5fc5c4fd-aaba-4149-86aa-ded1a82da601/serve-healthcheck-canary/0.log"
Jul 03 09:47:47.245323 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:47.245284    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:47.423130 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:47.423087    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:47.533309 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:47.533216    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:47.650859 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:47.650794    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:47.726874 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:47.726826    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wpk2r_7bcdf3e0-7d30-4010-9219-8a8901e43f25/kube-storage-version-migrator-operator/0.log"
Jul 03 09:47:47.749717 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:47.749684    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:47.965359 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:47.965324    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:48.248156 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:48.248069    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:48.934364 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:48.934334    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:49.337139 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:49.337060    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:49.538047 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:49.538015    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:49.736074 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:49.736041    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:50.135559 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:50.135474    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:50.156780 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:50.156743    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:50.156922 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:50.156812    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:50.157011 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:50.156991    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 09:47:50.157084 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:50.157073    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls podName:7f1108aa-d8c6-4608-aa94-a85740bfdc1e nodeName:}" failed. No retries permitted until 2026-07-03 09:47:58.157051895 +0000 UTC m=+280.022340544 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gz8ww" (UID: "7f1108aa-d8c6-4608-aa94-a85740bfdc1e") : secret "insights-runtime-extractor-tls" not found
Jul 03 09:47:50.157292 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:50.157273    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:50.727348 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:50.727313    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:51.043091 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:51.043012    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:51.143461 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:51.143430    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:51.327737 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:51.327643    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:51.531882 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:51.531824    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:51.728443 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:51.728405    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:52.131917 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:52.131825    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:52.375954 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:52.375921    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:52.376117 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:52.375963    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:52.376117 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:47:52.376100    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle podName:865a12bb-5687-4bc6-8f85-8a5df5db37fa nodeName:}" failed. No retries permitted until 2026-07-03 09:48:08.376070964 +0000 UTC m=+290.241359598 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle") pod "router-default-c5cd9c95-m6fvt" (UID: "865a12bb-5687-4bc6-8f85-8a5df5db37fa") : configmap references non-existent config key: service-ca.crt
Jul 03 09:47:52.378230 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:52.378212    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/865a12bb-5687-4bc6-8f85-8a5df5db37fa-metrics-certs\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:47:52.927174 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:52.927141    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:53.327227 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:53.327149    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:53.534454 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:53.534418    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:53.727245 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:53.727207    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:54.005377 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:54.005293    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:54.129206 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:54.129175    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:54.728549 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:54.728514    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:55.329014 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:55.328983    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:55.533821 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:55.533789    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:55.728776 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:55.728730    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:55.948496 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:55.948460    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:56.130638 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:56.130551    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:56.536081 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:56.536049    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:57.142691 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:57.142659    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:57.533981 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:57.533953    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:57.731499 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:57.731468    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:57.934964 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:57.934927    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:47:58.223721 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:58.223628    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:58.225947 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:58.225923    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f1108aa-d8c6-4608-aa94-a85740bfdc1e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gz8ww\" (UID: \"7f1108aa-d8c6-4608-aa94-a85740bfdc1e\") " pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:58.245951 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:58.245916    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gz8ww"
Jul 03 09:47:58.372639 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:58.372605    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gz8ww"]
Jul 03 09:47:58.375389 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:47:58.375358    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f1108aa_d8c6_4608_aa94_a85740bfdc1e.slice/crio-bee53eedac9a7a91653418ff5b1c4ace7dff9bb9d5b2d82d8af9c06a910f09cc WatchSource:0}: Error finding container bee53eedac9a7a91653418ff5b1c4ace7dff9bb9d5b2d82d8af9c06a910f09cc: Status 404 returned error can't find the container with id bee53eedac9a7a91653418ff5b1c4ace7dff9bb9d5b2d82d8af9c06a910f09cc
Jul 03 09:47:58.652788 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:47:58.652752    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gz8ww" event={"ID":"7f1108aa-d8c6-4608-aa94-a85740bfdc1e","Type":"ContainerStarted","Data":"bee53eedac9a7a91653418ff5b1c4ace7dff9bb9d5b2d82d8af9c06a910f09cc"}
Jul 03 09:48:00.743268 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:00.743238    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:00.806617 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:00.806581    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:01.318799 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:48:01.318755    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-wbkqf" podUID="889943a6-a12d-4476-ba1e-d0fcc7de0f42"
Jul 03 09:48:01.661563 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:01.661531    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:48:02.665804 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:02.665770    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gz8ww" event={"ID":"7f1108aa-d8c6-4608-aa94-a85740bfdc1e","Type":"ContainerStarted","Data":"083ea6471f30f6b2d18576f7f245dd92f250cc9e1d887ba4ed2894915e3950bb"}
Jul 03 09:48:03.671033 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:03.670989    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gz8ww" event={"ID":"7f1108aa-d8c6-4608-aa94-a85740bfdc1e","Type":"ContainerStarted","Data":"3e43940ed73f1a7b057f5eba8ed0562979fcf5500799755f8e6c7b28b7090e7c"}
Jul 03 09:48:04.377894 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:04.377853    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:48:04.380361 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:04.380337    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/889943a6-a12d-4476-ba1e-d0fcc7de0f42-metrics-tls\") pod \"dns-default-wbkqf\" (UID: \"889943a6-a12d-4476-ba1e-d0fcc7de0f42\") " pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:48:04.664650 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:04.664617    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7ckvl\""
Jul 03 09:48:04.672086 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:04.672038    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:48:05.230496 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:05.230464    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wbkqf"]
Jul 03 09:48:05.233551 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:48:05.233529    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889943a6_a12d_4476_ba1e_d0fcc7de0f42.slice/crio-6785f89fae5b87116f5143f83611617c5461f0327b338745ceca050822c2fcab WatchSource:0}: Error finding container 6785f89fae5b87116f5143f83611617c5461f0327b338745ceca050822c2fcab: Status 404 returned error can't find the container with id 6785f89fae5b87116f5143f83611617c5461f0327b338745ceca050822c2fcab
Jul 03 09:48:05.680240 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:05.680198    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gz8ww" event={"ID":"7f1108aa-d8c6-4608-aa94-a85740bfdc1e","Type":"ContainerStarted","Data":"561ce7ae8322e03a7ec686340c7dc4d5c62a0e992f43374a65e0c39d40fc95dd"}
Jul 03 09:48:05.681510 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:05.681484    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wbkqf" event={"ID":"889943a6-a12d-4476-ba1e-d0fcc7de0f42","Type":"ContainerStarted","Data":"6785f89fae5b87116f5143f83611617c5461f0327b338745ceca050822c2fcab"}
Jul 03 09:48:05.699088 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:05.699044    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gz8ww" podStartSLOduration=17.029557519 podStartE2EDuration="23.699028259s" totalImagesPullingTime="6.66947074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 09:47:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:48:05.697979665 +0000 UTC m=+287.563268366" watchObservedRunningTime="2026-07-03 09:48:05.699028259 +0000 UTC m=+287.564316917"
Jul 03 09:48:07.690344 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:07.690307    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wbkqf" event={"ID":"889943a6-a12d-4476-ba1e-d0fcc7de0f42","Type":"ContainerStarted","Data":"9feeda0f63c8353db293636dcb0e33663af8fb41d7d7db9af600f62257215b11"}
Jul 03 09:48:07.690762 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:07.690363    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wbkqf" event={"ID":"889943a6-a12d-4476-ba1e-d0fcc7de0f42","Type":"ContainerStarted","Data":"f6fd6953e3f42c15df96e35074ac7ff6d8d6042c6cb2fba53af1eac6a47a2f04"}
Jul 03 09:48:07.690762 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:07.690399    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:48:07.709983 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:07.709939    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wbkqf" podStartSLOduration=252.209664781 podStartE2EDuration="4m13.709922737s" totalImagesPullingTime="1.500257956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:43:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:48:07.708924767 +0000 UTC m=+289.574213423" watchObservedRunningTime="2026-07-03 09:48:07.709922737 +0000 UTC m=+289.575211392"
Jul 03 09:48:07.910477 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:07.910440    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-597978765c-8khg5"]
Jul 03 09:48:07.910668 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:48:07.910650    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-597978765c-8khg5" podUID="0c15a693-df72-4df6-84ee-0084b4c4de43"
Jul 03 09:48:08.045271 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.045165    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:08.141049 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.141018    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:08.238006 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.237977    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:08.411338 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.411305    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:48:08.411954 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.411935    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/865a12bb-5687-4bc6-8f85-8a5df5db37fa-service-ca-bundle\") pod \"router-default-c5cd9c95-m6fvt\" (UID: \"865a12bb-5687-4bc6-8f85-8a5df5db37fa\") " pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:48:08.653710 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.653665    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:48:08.694094 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.694060    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:48:08.701021 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.700993    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:48:08.784490 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.784463    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c5cd9c95-m6fvt"]
Jul 03 09:48:08.787423 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:48:08.787399    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod865a12bb_5687_4bc6_8f85_8a5df5db37fa.slice/crio-40051083f1e653523c0f5e0f27d683bcef7133a4168c71f3514f302bbc6cbc74 WatchSource:0}: Error finding container 40051083f1e653523c0f5e0f27d683bcef7133a4168c71f3514f302bbc6cbc74: Status 404 returned error can't find the container with id 40051083f1e653523c0f5e0f27d683bcef7133a4168c71f3514f302bbc6cbc74
Jul 03 09:48:08.814059 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.814036    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c15a693-df72-4df6-84ee-0084b4c4de43-trusted-ca\") pod \"0c15a693-df72-4df6-84ee-0084b4c4de43\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") "
Jul 03 09:48:08.814166 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.814074    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-bound-sa-token\") pod \"0c15a693-df72-4df6-84ee-0084b4c4de43\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") "
Jul 03 09:48:08.814166 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.814112    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj87x\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-kube-api-access-pj87x\") pod \"0c15a693-df72-4df6-84ee-0084b4c4de43\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") "
Jul 03 09:48:08.814166 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.814128    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-certificates\") pod \"0c15a693-df72-4df6-84ee-0084b4c4de43\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") "
Jul 03 09:48:08.814166 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.814160    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c15a693-df72-4df6-84ee-0084b4c4de43-installation-pull-secrets\") pod \"0c15a693-df72-4df6-84ee-0084b4c4de43\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") "
Jul 03 09:48:08.814338 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.814194    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c15a693-df72-4df6-84ee-0084b4c4de43-image-registry-private-configuration\") pod \"0c15a693-df72-4df6-84ee-0084b4c4de43\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") "
Jul 03 09:48:08.814338 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.814225    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c15a693-df72-4df6-84ee-0084b4c4de43-ca-trust-extracted\") pod \"0c15a693-df72-4df6-84ee-0084b4c4de43\" (UID: \"0c15a693-df72-4df6-84ee-0084b4c4de43\") "
Jul 03 09:48:08.814508 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.814481    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c15a693-df72-4df6-84ee-0084b4c4de43-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0c15a693-df72-4df6-84ee-0084b4c4de43" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 09:48:08.814663 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.814637    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0c15a693-df72-4df6-84ee-0084b4c4de43" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 09:48:08.815087 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.814914    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c15a693-df72-4df6-84ee-0084b4c4de43-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0c15a693-df72-4df6-84ee-0084b4c4de43" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 09:48:08.816533 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.816508    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c15a693-df72-4df6-84ee-0084b4c4de43-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0c15a693-df72-4df6-84ee-0084b4c4de43" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 09:48:08.816619 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.816576    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c15a693-df72-4df6-84ee-0084b4c4de43-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0c15a693-df72-4df6-84ee-0084b4c4de43" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 09:48:08.816713 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.816693    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-kube-api-access-pj87x" (OuterVolumeSpecName: "kube-api-access-pj87x") pod "0c15a693-df72-4df6-84ee-0084b4c4de43" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43"). InnerVolumeSpecName "kube-api-access-pj87x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:48:08.816755 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.816736    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0c15a693-df72-4df6-84ee-0084b4c4de43" (UID: "0c15a693-df72-4df6-84ee-0084b4c4de43"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:48:08.915164 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.915076    2571 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c15a693-df72-4df6-84ee-0084b4c4de43-installation-pull-secrets\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:48:08.915164 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.915110    2571 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c15a693-df72-4df6-84ee-0084b4c4de43-image-registry-private-configuration\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:48:08.915164 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.915120    2571 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c15a693-df72-4df6-84ee-0084b4c4de43-ca-trust-extracted\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:48:08.915164 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.915132    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c15a693-df72-4df6-84ee-0084b4c4de43-trusted-ca\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:48:08.915164 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.915141    2571 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-bound-sa-token\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:48:08.915164 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.915149    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj87x\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-kube-api-access-pj87x\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:48:08.915164 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:08.915158    2571 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-certificates\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:48:09.698783 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:09.698740    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c5cd9c95-m6fvt" event={"ID":"865a12bb-5687-4bc6-8f85-8a5df5db37fa","Type":"ContainerStarted","Data":"43c1243f3c68af6c978f60520eacabb45cf73f6a1e9425599eb62c447cb5564b"}
Jul 03 09:48:09.699308 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:09.698823    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c5cd9c95-m6fvt" event={"ID":"865a12bb-5687-4bc6-8f85-8a5df5db37fa","Type":"ContainerStarted","Data":"40051083f1e653523c0f5e0f27d683bcef7133a4168c71f3514f302bbc6cbc74"}
Jul 03 09:48:09.699308 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:09.698759    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-597978765c-8khg5"
Jul 03 09:48:09.718767 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:09.718698    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-c5cd9c95-m6fvt" podStartSLOduration=33.718682185 podStartE2EDuration="33.718682185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:47:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:48:09.717423673 +0000 UTC m=+291.582712328" watchObservedRunningTime="2026-07-03 09:48:09.718682185 +0000 UTC m=+291.583970843"
Jul 03 09:48:09.744228 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:09.744197    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-597978765c-8khg5"]
Jul 03 09:48:09.748392 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:09.748367    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-597978765c-8khg5"]
Jul 03 09:48:09.821152 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:09.821106    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c15a693-df72-4df6-84ee-0084b4c4de43-registry-tls\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:48:10.654466 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:10.654426    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:48:10.657171 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:10.657147    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:48:10.702440 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:10.702409    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:48:10.703703 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:10.703680    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-c5cd9c95-m6fvt"
Jul 03 09:48:10.763813 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:10.763781    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c15a693-df72-4df6-84ee-0084b4c4de43" path="/var/lib/kubelet/pods/0c15a693-df72-4df6-84ee-0084b4c4de43/volumes"
Jul 03 09:48:10.880871 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:10.880819    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:10.982569 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:10.982501    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:11.037993 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:11.037961    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf"]
Jul 03 09:48:11.041160 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:11.041137    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf"
Jul 03 09:48:11.043620 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:11.043592    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8zwwq\""
Jul 03 09:48:11.044059 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:11.044042    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 03 09:48:11.048959 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:11.048935    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf"]
Jul 03 09:48:11.056011 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:11.055988    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:11.129802 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:11.129761    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c2584a2c-1655-4547-91a1-ef907627535a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-q66kf\" (UID: \"c2584a2c-1655-4547-91a1-ef907627535a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf"
Jul 03 09:48:11.231023 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:11.230983    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c2584a2c-1655-4547-91a1-ef907627535a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-q66kf\" (UID: \"c2584a2c-1655-4547-91a1-ef907627535a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf"
Jul 03 09:48:11.231197 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:48:11.231131    2571 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 03 09:48:11.231242 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:48:11.231198    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2584a2c-1655-4547-91a1-ef907627535a-tls-certificates podName:c2584a2c-1655-4547-91a1-ef907627535a nodeName:}" failed. No retries permitted until 2026-07-03 09:48:11.731180044 +0000 UTC m=+293.596468689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/c2584a2c-1655-4547-91a1-ef907627535a-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-q66kf" (UID: "c2584a2c-1655-4547-91a1-ef907627535a") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 03 09:48:11.295782 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:11.295706    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:11.736133 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:11.736099    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c2584a2c-1655-4547-91a1-ef907627535a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-q66kf\" (UID: \"c2584a2c-1655-4547-91a1-ef907627535a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf"
Jul 03 09:48:11.738551 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:11.738525    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c2584a2c-1655-4547-91a1-ef907627535a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-q66kf\" (UID: \"c2584a2c-1655-4547-91a1-ef907627535a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf"
Jul 03 09:48:11.949942 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:11.949914    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf"
Jul 03 09:48:12.073846 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:12.073800    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf"]
Jul 03 09:48:12.076432 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:48:12.076406    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2584a2c_1655_4547_91a1_ef907627535a.slice/crio-e0daefef223f8ee20bfa337dc2a956ab76502aa2236b9ebdbf4dce70709c47d6 WatchSource:0}: Error finding container e0daefef223f8ee20bfa337dc2a956ab76502aa2236b9ebdbf4dce70709c47d6: Status 404 returned error can't find the container with id e0daefef223f8ee20bfa337dc2a956ab76502aa2236b9ebdbf4dce70709c47d6
Jul 03 09:48:12.710349 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:12.710303    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf" event={"ID":"c2584a2c-1655-4547-91a1-ef907627535a","Type":"ContainerStarted","Data":"e0daefef223f8ee20bfa337dc2a956ab76502aa2236b9ebdbf4dce70709c47d6"}
Jul 03 09:48:13.714591 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:13.714553    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf" event={"ID":"c2584a2c-1655-4547-91a1-ef907627535a","Type":"ContainerStarted","Data":"ffaf10b6a70700d48999da486121f496de937f3d46f6ceebb5cfa8bad202f7c5"}
Jul 03 09:48:13.715076 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:13.714765    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf"
Jul 03 09:48:13.719361 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:13.719342    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf"
Jul 03 09:48:13.734686 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:13.734637    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-q66kf" podStartSLOduration=1.54582678 podStartE2EDuration="2.734624582s" totalImagesPullingTime="1.188797802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:48:13.733663371 +0000 UTC m=+295.598952027" watchObservedRunningTime="2026-07-03 09:48:13.734624582 +0000 UTC m=+295.599913237"
Jul 03 09:48:14.062914 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.062813    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:14.093502 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.092742    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"]
Jul 03 09:48:14.108856 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.102237    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.108856 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.105171    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 03 09:48:14.108856 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.105429    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-vlwf5\""
Jul 03 09:48:14.108856 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.105453    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 03 09:48:14.108856 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.106331    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 03 09:48:14.108856 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.106569    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 03 09:48:14.108856 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.106747    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 03 09:48:14.120533 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.116561    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"]
Jul 03 09:48:14.123884 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.123852    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:14.154319 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.154282    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm5ds\" (UniqueName: \"kubernetes.io/projected/34415362-51c1-41fd-a525-7520f25f65e8-kube-api-access-hm5ds\") pod \"prometheus-operator-567f7b8b7c-zvhxp\" (UID: \"34415362-51c1-41fd-a525-7520f25f65e8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.154479 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.154340    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34415362-51c1-41fd-a525-7520f25f65e8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-zvhxp\" (UID: \"34415362-51c1-41fd-a525-7520f25f65e8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.154479 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.154405    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34415362-51c1-41fd-a525-7520f25f65e8-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-zvhxp\" (UID: \"34415362-51c1-41fd-a525-7520f25f65e8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.154479 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.154445    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34415362-51c1-41fd-a525-7520f25f65e8-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-zvhxp\" (UID: \"34415362-51c1-41fd-a525-7520f25f65e8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.190652 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.190627    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:14.255791 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.255752    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm5ds\" (UniqueName: \"kubernetes.io/projected/34415362-51c1-41fd-a525-7520f25f65e8-kube-api-access-hm5ds\") pod \"prometheus-operator-567f7b8b7c-zvhxp\" (UID: \"34415362-51c1-41fd-a525-7520f25f65e8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.255996 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.255812    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34415362-51c1-41fd-a525-7520f25f65e8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-zvhxp\" (UID: \"34415362-51c1-41fd-a525-7520f25f65e8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.255996 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.255872    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34415362-51c1-41fd-a525-7520f25f65e8-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-zvhxp\" (UID: \"34415362-51c1-41fd-a525-7520f25f65e8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.255996 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.255911    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34415362-51c1-41fd-a525-7520f25f65e8-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-zvhxp\" (UID: \"34415362-51c1-41fd-a525-7520f25f65e8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.256602 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.256577    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34415362-51c1-41fd-a525-7520f25f65e8-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-zvhxp\" (UID: \"34415362-51c1-41fd-a525-7520f25f65e8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.258257 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.258231    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34415362-51c1-41fd-a525-7520f25f65e8-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-zvhxp\" (UID: \"34415362-51c1-41fd-a525-7520f25f65e8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.258352 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.258310    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34415362-51c1-41fd-a525-7520f25f65e8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-zvhxp\" (UID: \"34415362-51c1-41fd-a525-7520f25f65e8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.263998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.263974    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm5ds\" (UniqueName: \"kubernetes.io/projected/34415362-51c1-41fd-a525-7520f25f65e8-kube-api-access-hm5ds\") pod \"prometheus-operator-567f7b8b7c-zvhxp\" (UID: \"34415362-51c1-41fd-a525-7520f25f65e8\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.429932 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.429895    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"
Jul 03 09:48:14.555072 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.555042    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp"]
Jul 03 09:48:14.558118 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:48:14.558090    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34415362_51c1_41fd_a525_7520f25f65e8.slice/crio-2302363e9e906cd4265e7ca60ccfecb6b12bac34ba27c35b801938dd7846e84b WatchSource:0}: Error finding container 2302363e9e906cd4265e7ca60ccfecb6b12bac34ba27c35b801938dd7846e84b: Status 404 returned error can't find the container with id 2302363e9e906cd4265e7ca60ccfecb6b12bac34ba27c35b801938dd7846e84b
Jul 03 09:48:14.719403 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:14.719310    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp" event={"ID":"34415362-51c1-41fd-a525-7520f25f65e8","Type":"ContainerStarted","Data":"2302363e9e906cd4265e7ca60ccfecb6b12bac34ba27c35b801938dd7846e84b"}
Jul 03 09:48:14.758848 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:48:14.758797    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-txhn2" podUID="f18ac135-e44e-413a-9bda-248eacb6287b"
Jul 03 09:48:15.743385 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:15.743349    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:15.844323 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:15.844292    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:16.029301 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:16.029216    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:16.728399 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:16.728366    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp" event={"ID":"34415362-51c1-41fd-a525-7520f25f65e8","Type":"ContainerStarted","Data":"e47dbf2afcf0b61024cf79fcae9e269e2e05eebac30f203ddf657cf7cff75832"}
Jul 03 09:48:16.728566 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:16.728416    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp" event={"ID":"34415362-51c1-41fd-a525-7520f25f65e8","Type":"ContainerStarted","Data":"a46eb38e69733da55c17e02f402e2db5d7b448326421d05bd791c0f2179cfa02"}
Jul 03 09:48:16.746123 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:16.746077    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-zvhxp" podStartSLOduration=1.524765605 podStartE2EDuration="2.74606312s" totalImagesPullingTime="1.221297515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:48:16.746022291 +0000 UTC m=+298.611310948" watchObservedRunningTime="2026-07-03 09:48:16.74606312 +0000 UTC m=+298.611351774"
Jul 03 09:48:17.696516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:17.696490    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wbkqf"
Jul 03 09:48:18.206500 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.206467    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:18.344379 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.344345    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:18.461471 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.461390    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"]
Jul 03 09:48:18.461471 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.461431    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:18.465323 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.465296    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:18.468555 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.468526    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-nbfkd\""
Jul 03 09:48:18.468781 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.468764    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 03 09:48:18.468973 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.468950    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 03 09:48:18.479130 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.479107    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"]
Jul 03 09:48:18.483306 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.483284    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dgqxn"]
Jul 03 09:48:18.486340 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.486323    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.488937 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.488919    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w64h7\""
Jul 03 09:48:18.489045 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.488920    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 03 09:48:18.489045 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.488922    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 03 09:48:18.489158 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.488973    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 03 09:48:18.590347 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.590314    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91401011-fe57-4baf-80d2-2d4e813fb10e-sys\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.590347 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.590354    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-lc59n\" (UID: \"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:18.590580 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.590380    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-tls\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.590580 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.590405    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91401011-fe57-4baf-80d2-2d4e813fb10e-metrics-client-ca\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.590580 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.590426    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-wtmp\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.590580 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.590443    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-lc59n\" (UID: \"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:18.590580 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.590464    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-accelerators-collector-config\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.590580 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.590493    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.590580 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.590575    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzzfr\" (UniqueName: \"kubernetes.io/projected/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-kube-api-access-hzzfr\") pod \"openshift-state-metrics-54cf96dfcf-lc59n\" (UID: \"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:18.590956 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.590605    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-textfile\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.590956 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.590630    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqx87\" (UniqueName: \"kubernetes.io/projected/91401011-fe57-4baf-80d2-2d4e813fb10e-kube-api-access-sqx87\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.590956 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.590667    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-lc59n\" (UID: \"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:18.590956 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.590697    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/91401011-fe57-4baf-80d2-2d4e813fb10e-root\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.596624 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.596596    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 09:48:18.597283 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.597088    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 09:48:18.599773 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.599755    2571 kubelet.go:1648] "Image garbage collection succeeded"
Jul 03 09:48:18.691968 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.691934    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.692131 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692008    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzzfr\" (UniqueName: \"kubernetes.io/projected/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-kube-api-access-hzzfr\") pod \"openshift-state-metrics-54cf96dfcf-lc59n\" (UID: \"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:18.692131 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692037    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-textfile\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.692131 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692063    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqx87\" (UniqueName: \"kubernetes.io/projected/91401011-fe57-4baf-80d2-2d4e813fb10e-kube-api-access-sqx87\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.692131 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692096    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-lc59n\" (UID: \"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:18.692131 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692126    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/91401011-fe57-4baf-80d2-2d4e813fb10e-root\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.692397 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692173    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91401011-fe57-4baf-80d2-2d4e813fb10e-sys\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.692397 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692198    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-lc59n\" (UID: \"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:18.692397 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692223    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-tls\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.692397 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692246    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91401011-fe57-4baf-80d2-2d4e813fb10e-metrics-client-ca\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.692397 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692271    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-wtmp\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.692397 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692297    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-lc59n\" (UID: \"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:18.692397 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692324    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-accelerators-collector-config\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.692397 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692385    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91401011-fe57-4baf-80d2-2d4e813fb10e-sys\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.692987 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.692964    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-textfile\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.693141 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.693121    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91401011-fe57-4baf-80d2-2d4e813fb10e-metrics-client-ca\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.693280 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.693265    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-wtmp\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.693337 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.693294    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/91401011-fe57-4baf-80d2-2d4e813fb10e-root\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.693904 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.693886    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-lc59n\" (UID: \"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:18.694881 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.694860    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 03 09:48:18.695308 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.695291    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 03 09:48:18.695883 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.695522    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 03 09:48:18.695883 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.695870    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 03 09:48:18.696031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.695884    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 03 09:48:18.702179 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.702048    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqx87\" (UniqueName: \"kubernetes.io/projected/91401011-fe57-4baf-80d2-2d4e813fb10e-kube-api-access-sqx87\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.702401 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.702175    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzzfr\" (UniqueName: \"kubernetes.io/projected/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-kube-api-access-hzzfr\") pod \"openshift-state-metrics-54cf96dfcf-lc59n\" (UID: \"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:18.702955 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.702931    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-accelerators-collector-config\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.704780 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:48:18.704540    2571 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 03 09:48:18.704780 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:48:18.704617    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-tls podName:91401011-fe57-4baf-80d2-2d4e813fb10e nodeName:}" failed. No retries permitted until 2026-07-03 09:48:19.204595182 +0000 UTC m=+301.069883854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-tls") pod "node-exporter-dgqxn" (UID: "91401011-fe57-4baf-80d2-2d4e813fb10e") : secret "node-exporter-tls" not found
Jul 03 09:48:18.704780 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.704671    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:18.704780 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:48:18.704546    2571 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 03 09:48:18.704780 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:48:18.704751    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-openshift-state-metrics-tls podName:d5f267cc-926a-4c7d-bcf6-f1aad226c0d4 nodeName:}" failed. No retries permitted until 2026-07-03 09:48:19.20473315 +0000 UTC m=+301.070021789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-lc59n" (UID: "d5f267cc-926a-4c7d-bcf6-f1aad226c0d4") : secret "openshift-state-metrics-tls" not found
Jul 03 09:48:18.707383 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:18.707357    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-lc59n\" (UID: \"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:19.296941 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.296901    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-lc59n\" (UID: \"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:19.296941 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.296942    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-tls\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:19.299215 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.299189    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/91401011-fe57-4baf-80d2-2d4e813fb10e-node-exporter-tls\") pod \"node-exporter-dgqxn\" (UID: \"91401011-fe57-4baf-80d2-2d4e813fb10e\") " pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:19.299301 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.299239    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d5f267cc-926a-4c7d-bcf6-f1aad226c0d4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-lc59n\" (UID: \"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:19.380598 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.380512    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-nbfkd\""
Jul 03 09:48:19.388683 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.388653    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"
Jul 03 09:48:19.400764 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.400736    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w64h7\""
Jul 03 09:48:19.408297 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.408270    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dgqxn"
Jul 03 09:48:19.422338 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:48:19.422309    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91401011_fe57_4baf_80d2_2d4e813fb10e.slice/crio-60d6e77e7d7d5093b782c3858de61ad29736afdc33e06643b3d7a8e688364859 WatchSource:0}: Error finding container 60d6e77e7d7d5093b782c3858de61ad29736afdc33e06643b3d7a8e688364859: Status 404 returned error can't find the container with id 60d6e77e7d7d5093b782c3858de61ad29736afdc33e06643b3d7a8e688364859
Jul 03 09:48:19.529329 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.529290    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n"]
Jul 03 09:48:19.533659 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:48:19.533631    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5f267cc_926a_4c7d_bcf6_f1aad226c0d4.slice/crio-4d832794bed778bd1397a02fce2a1b3fa2c6a004009722875893a9ca493523e4 WatchSource:0}: Error finding container 4d832794bed778bd1397a02fce2a1b3fa2c6a004009722875893a9ca493523e4: Status 404 returned error can't find the container with id 4d832794bed778bd1397a02fce2a1b3fa2c6a004009722875893a9ca493523e4
Jul 03 09:48:19.738565 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.738531    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dgqxn" event={"ID":"91401011-fe57-4baf-80d2-2d4e813fb10e","Type":"ContainerStarted","Data":"60d6e77e7d7d5093b782c3858de61ad29736afdc33e06643b3d7a8e688364859"}
Jul 03 09:48:19.740459 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.740434    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n" event={"ID":"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4","Type":"ContainerStarted","Data":"a81dbf369fffc82ef11184ea1f8ed411d37aee2c35847b78a2aecd2cecb66f1a"}
Jul 03 09:48:19.740578 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.740478    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n" event={"ID":"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4","Type":"ContainerStarted","Data":"430e6ee4639845244598886fdfc8d7a46272dc52b989f6fb92b927ad64b615ab"}
Jul 03 09:48:19.740578 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.740496    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n" event={"ID":"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4","Type":"ContainerStarted","Data":"4d832794bed778bd1397a02fce2a1b3fa2c6a004009722875893a9ca493523e4"}
Jul 03 09:48:19.743403 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:19.743379    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:20.442206 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.442171    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"]
Jul 03 09:48:20.451843 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.451801    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.454910 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.454886    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 03 09:48:20.455383 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.454944    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rnxfs\""
Jul 03 09:48:20.455383 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.455102    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 03 09:48:20.455516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.455470    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 03 09:48:20.456005 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.455981    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 03 09:48:20.456247 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.456229    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 03 09:48:20.456692 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.456658    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6u0r1r0ib7v1e\""
Jul 03 09:48:20.460295 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.460275    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"]
Jul 03 09:48:20.607431 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.606867    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-tls\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.607431 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.606915    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.607431 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.606943    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvshs\" (UniqueName: \"kubernetes.io/projected/47d7cde5-4688-4610-abd4-de8ff576fd75-kube-api-access-wvshs\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.607431 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.607047    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-grpc-tls\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.607431 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.607089    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.607431 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.607121    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.607431 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.607152    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.607431 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.607182    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47d7cde5-4688-4610-abd4-de8ff576fd75-metrics-client-ca\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.707688 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.707662    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.707805 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.707706    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47d7cde5-4688-4610-abd4-de8ff576fd75-metrics-client-ca\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.707805 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.707776    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-tls\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.707920 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.707808    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.707920 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.707851    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvshs\" (UniqueName: \"kubernetes.io/projected/47d7cde5-4688-4610-abd4-de8ff576fd75-kube-api-access-wvshs\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.707920 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.707897    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-grpc-tls\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.708074 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.707932    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.708074 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.707966    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.708988 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.708951    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47d7cde5-4688-4610-abd4-de8ff576fd75-metrics-client-ca\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.710588 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.710540    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.710908 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.710859    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-grpc-tls\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.711205 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.711042    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.711288 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.711205    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.711487 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.711466    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-tls\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.711568 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.711550    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47d7cde5-4688-4610-abd4-de8ff576fd75-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.718012 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.717981    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvshs\" (UniqueName: \"kubernetes.io/projected/47d7cde5-4688-4610-abd4-de8ff576fd75-kube-api-access-wvshs\") pod \"thanos-querier-596fdd74fc-lgvlc\" (UID: \"47d7cde5-4688-4610-abd4-de8ff576fd75\") " pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.746971 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.746874    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dgqxn" event={"ID":"91401011-fe57-4baf-80d2-2d4e813fb10e","Type":"ContainerStarted","Data":"c8e2b14b01d5bb994ad7026e290a30d15ee8e25844909df9d847b3dd4c60cc32"}
Jul 03 09:48:20.749336 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.749299    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n" event={"ID":"d5f267cc-926a-4c7d-bcf6-f1aad226c0d4","Type":"ContainerStarted","Data":"9626ca3b2862a41681fa6bde41050361ecfb7bfe3e5803683481b8e1d005161e"}
Jul 03 09:48:20.764628 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.764603    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:20.803085 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.803034    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-lc59n" podStartSLOduration=1.82333501 podStartE2EDuration="2.803014472s" totalImagesPullingTime="979.679462ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:48:20.802194776 +0000 UTC m=+302.667483431" watchObservedRunningTime="2026-07-03 09:48:20.803014472 +0000 UTC m=+302.668303130"
Jul 03 09:48:20.896129 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.896100    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"]
Jul 03 09:48:20.898390 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:48:20.898362    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d7cde5_4688_4610_abd4_de8ff576fd75.slice/crio-f635df509cc26db23979d4f45bfb4b0470778e6c06817869ece0a2acb1a40b2b WatchSource:0}: Error finding container f635df509cc26db23979d4f45bfb4b0470778e6c06817869ece0a2acb1a40b2b: Status 404 returned error can't find the container with id f635df509cc26db23979d4f45bfb4b0470778e6c06817869ece0a2acb1a40b2b
Jul 03 09:48:20.900562 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:20.900544    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 09:48:21.383275 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:21.383234    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:21.485260 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:21.485229    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:21.585904 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:21.585871    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:21.685537 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:21.685502    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:21.754169 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:21.754134    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc" event={"ID":"47d7cde5-4688-4610-abd4-de8ff576fd75","Type":"ContainerStarted","Data":"f635df509cc26db23979d4f45bfb4b0470778e6c06817869ece0a2acb1a40b2b"}
Jul 03 09:48:21.755534 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:21.755510    2571 generic.go:356] "Generic (PLEG): container finished" podID="91401011-fe57-4baf-80d2-2d4e813fb10e" containerID="c8e2b14b01d5bb994ad7026e290a30d15ee8e25844909df9d847b3dd4c60cc32" exitCode=0
Jul 03 09:48:21.755677 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:21.755595    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dgqxn" event={"ID":"91401011-fe57-4baf-80d2-2d4e813fb10e","Type":"ContainerDied","Data":"c8e2b14b01d5bb994ad7026e290a30d15ee8e25844909df9d847b3dd4c60cc32"}
Jul 03 09:48:22.081204 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.081115    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:22.183292 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.183258    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:22.252285 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.252249    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:22.749259 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.749206    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"]
Jul 03 09:48:22.752847 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.752813    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.755157 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.755138    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 03 09:48:22.756447 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.756426    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-gvh5p\""
Jul 03 09:48:22.756552 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.756444    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 03 09:48:22.756552 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.756465    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-9l1hnjcgm5e44\""
Jul 03 09:48:22.756552 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.756541    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 03 09:48:22.756690 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.756622    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 03 09:48:22.774341 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.774309    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"]
Jul 03 09:48:22.774494 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.774366    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dgqxn" event={"ID":"91401011-fe57-4baf-80d2-2d4e813fb10e","Type":"ContainerStarted","Data":"c6eca7115e591b45a9a7aeea00fa82fa9e968053f972b01e27c9742c098879c9"}
Jul 03 09:48:22.774494 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.774409    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dgqxn" event={"ID":"91401011-fe57-4baf-80d2-2d4e813fb10e","Type":"ContainerStarted","Data":"8db20cb8386ccd17c31f340f0c96c40c2dfaf04467ea6230e1469c19d3d2b2de"}
Jul 03 09:48:22.798314 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.798265    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dgqxn" podStartSLOduration=3.556364084 podStartE2EDuration="4.798249771s" totalImagesPullingTime="1.172077268s" totalInitContainerRuntime="69.808419ms" isStatefulPod=true podCreationTimestamp="2026-07-03 09:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:48:22.797462503 +0000 UTC m=+304.662751158" watchObservedRunningTime="2026-07-03 09:48:22.798249771 +0000 UTC m=+304.663538426"
Jul 03 09:48:22.829402 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.829366    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93117215-25d2-4b44-8977-47a0363088ab-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.829402 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.829405    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/93117215-25d2-4b44-8977-47a0363088ab-metrics-server-audit-profiles\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.829604 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.829443    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/93117215-25d2-4b44-8977-47a0363088ab-audit-log\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.829604 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.829485    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93117215-25d2-4b44-8977-47a0363088ab-client-ca-bundle\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.829604 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.829534    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/93117215-25d2-4b44-8977-47a0363088ab-secret-metrics-server-client-certs\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.829604 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.829563    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/93117215-25d2-4b44-8977-47a0363088ab-secret-metrics-server-tls\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.829604 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.829588    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7wtd\" (UniqueName: \"kubernetes.io/projected/93117215-25d2-4b44-8977-47a0363088ab-kube-api-access-w7wtd\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.930692 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.930651    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/93117215-25d2-4b44-8977-47a0363088ab-audit-log\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.930692 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.930690    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93117215-25d2-4b44-8977-47a0363088ab-client-ca-bundle\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.930943 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.930726    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/93117215-25d2-4b44-8977-47a0363088ab-secret-metrics-server-client-certs\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.930943 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.930749    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/93117215-25d2-4b44-8977-47a0363088ab-secret-metrics-server-tls\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.930943 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.930818    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7wtd\" (UniqueName: \"kubernetes.io/projected/93117215-25d2-4b44-8977-47a0363088ab-kube-api-access-w7wtd\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.930943 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.930896    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93117215-25d2-4b44-8977-47a0363088ab-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.930943 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.930928    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/93117215-25d2-4b44-8977-47a0363088ab-metrics-server-audit-profiles\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.931205 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.931119    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/93117215-25d2-4b44-8977-47a0363088ab-audit-log\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.932017 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.931990    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/93117215-25d2-4b44-8977-47a0363088ab-metrics-server-audit-profiles\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.932152 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.932019    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93117215-25d2-4b44-8977-47a0363088ab-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.933695 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.933671    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93117215-25d2-4b44-8977-47a0363088ab-client-ca-bundle\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.933919 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.933898    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/93117215-25d2-4b44-8977-47a0363088ab-secret-metrics-server-tls\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.934110 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.934091    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/93117215-25d2-4b44-8977-47a0363088ab-secret-metrics-server-client-certs\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:22.940072 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:22.940040    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7wtd\" (UniqueName: \"kubernetes.io/projected/93117215-25d2-4b44-8977-47a0363088ab-kube-api-access-w7wtd\") pod \"metrics-server-7966bfc5ff-8gnwj\" (UID: \"93117215-25d2-4b44-8977-47a0363088ab\") " pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:23.067581 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.067494    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:23.211891 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.211860    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"]
Jul 03 09:48:23.215347 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:48:23.215319    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93117215_25d2_4b44_8977_47a0363088ab.slice/crio-92b66146f3926ba0582403e86f0396a70433e3af3f55889ff3c43ece7a40ede0 WatchSource:0}: Error finding container 92b66146f3926ba0582403e86f0396a70433e3af3f55889ff3c43ece7a40ede0: Status 404 returned error can't find the container with id 92b66146f3926ba0582403e86f0396a70433e3af3f55889ff3c43ece7a40ede0
Jul 03 09:48:23.226649 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.226612    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:23.226988 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.226970    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"]
Jul 03 09:48:23.231298 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.231280    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"
Jul 03 09:48:23.233640 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.233622    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 03 09:48:23.233753 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.233700    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-gbhjq\""
Jul 03 09:48:23.241985 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.240501    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"]
Jul 03 09:48:23.335000 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.334906    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8b6a5cdc-691b-4621-bf36-133f421c85cc-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-77pdg\" (UID: \"8b6a5cdc-691b-4621-bf36-133f421c85cc\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"
Jul 03 09:48:23.335000 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.334982    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mkb5\" (UniqueName: \"kubernetes.io/projected/8b6a5cdc-691b-4621-bf36-133f421c85cc-kube-api-access-9mkb5\") pod \"monitoring-plugin-d5dcff7cf-77pdg\" (UID: \"8b6a5cdc-691b-4621-bf36-133f421c85cc\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"
Jul 03 09:48:23.436332 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.436289    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mkb5\" (UniqueName: \"kubernetes.io/projected/8b6a5cdc-691b-4621-bf36-133f421c85cc-kube-api-access-9mkb5\") pod \"monitoring-plugin-d5dcff7cf-77pdg\" (UID: \"8b6a5cdc-691b-4621-bf36-133f421c85cc\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"
Jul 03 09:48:23.436503 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.436369    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8b6a5cdc-691b-4621-bf36-133f421c85cc-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-77pdg\" (UID: \"8b6a5cdc-691b-4621-bf36-133f421c85cc\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"
Jul 03 09:48:23.436503 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:48:23.436485    2571 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 03 09:48:23.436627 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:48:23.436540    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b6a5cdc-691b-4621-bf36-133f421c85cc-monitoring-plugin-cert podName:8b6a5cdc-691b-4621-bf36-133f421c85cc nodeName:}" failed. No retries permitted until 2026-07-03 09:48:23.936522496 +0000 UTC m=+305.801811131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/8b6a5cdc-691b-4621-bf36-133f421c85cc-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-77pdg" (UID: "8b6a5cdc-691b-4621-bf36-133f421c85cc") : secret "monitoring-plugin-cert" not found
Jul 03 09:48:23.453385 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.453352    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mkb5\" (UniqueName: \"kubernetes.io/projected/8b6a5cdc-691b-4621-bf36-133f421c85cc-kube-api-access-9mkb5\") pod \"monitoring-plugin-d5dcff7cf-77pdg\" (UID: \"8b6a5cdc-691b-4621-bf36-133f421c85cc\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"
Jul 03 09:48:23.477236 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.477202    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:23.571188 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.571159    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:23.780196 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.780129    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj" event={"ID":"93117215-25d2-4b44-8977-47a0363088ab","Type":"ContainerStarted","Data":"92b66146f3926ba0582403e86f0396a70433e3af3f55889ff3c43ece7a40ede0"}
Jul 03 09:48:23.808309 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.808271    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:23.941876 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.941286    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8b6a5cdc-691b-4621-bf36-133f421c85cc-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-77pdg\" (UID: \"8b6a5cdc-691b-4621-bf36-133f421c85cc\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"
Jul 03 09:48:23.945281 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:23.945251    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8b6a5cdc-691b-4621-bf36-133f421c85cc-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-77pdg\" (UID: \"8b6a5cdc-691b-4621-bf36-133f421c85cc\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"
Jul 03 09:48:24.143516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:24.143415    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"
Jul 03 09:48:24.654473 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:24.654441    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:25.007074 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:25.007006    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"]
Jul 03 09:48:25.348558 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:25.348475    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:25.449420 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:25.449381    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:25.790592 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:25.790552    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc" event={"ID":"47d7cde5-4688-4610-abd4-de8ff576fd75","Type":"ContainerStarted","Data":"89d831f983c2fbdb5af9681d943197d0750e9fedc1aadf2f1e679e9c0ebada67"}
Jul 03 09:48:25.790769 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:25.790640    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc" event={"ID":"47d7cde5-4688-4610-abd4-de8ff576fd75","Type":"ContainerStarted","Data":"0376165782ae77c181f678843f9784e2a7e08c0d3ae7a9cf9f04de8a009652f0"}
Jul 03 09:48:25.790769 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:25.790681    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc" event={"ID":"47d7cde5-4688-4610-abd4-de8ff576fd75","Type":"ContainerStarted","Data":"96f8cf0790498b502e767998eb2b19f8a7e8c0e08f3d91a9612b54b88bf569f8"}
Jul 03 09:48:25.792619 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:25.792584    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj" event={"ID":"93117215-25d2-4b44-8977-47a0363088ab","Type":"ContainerStarted","Data":"541a33a2ec2203b7452e60dcb73ba71261d39bd55a34766988f4fe4a6411ac8e"}
Jul 03 09:48:25.793944 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:25.793907    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg" event={"ID":"8b6a5cdc-691b-4621-bf36-133f421c85cc","Type":"ContainerStarted","Data":"b3d44d09084a30aeb19147df34d39ba8a72783f774ec1b90e645a1f84de483ed"}
Jul 03 09:48:25.809732 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:25.809686    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj" podStartSLOduration=2.1853316 podStartE2EDuration="3.809670095s" totalImagesPullingTime="1.624338495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:48:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:48:25.808323663 +0000 UTC m=+307.673612318" watchObservedRunningTime="2026-07-03 09:48:25.809670095 +0000 UTC m=+307.674958749"
Jul 03 09:48:26.758734 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:26.758697    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:48:26.799140 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:26.799105    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg" event={"ID":"8b6a5cdc-691b-4621-bf36-133f421c85cc","Type":"ContainerStarted","Data":"bfcbf5718b36661c3c316debaf3e8067562a830f284ccaaac766803b9567387b"}
Jul 03 09:48:26.799344 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:26.799272    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"
Jul 03 09:48:26.802308 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:26.802282    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc" event={"ID":"47d7cde5-4688-4610-abd4-de8ff576fd75","Type":"ContainerStarted","Data":"a6cb6166ee681c4249a7fb9df8b3d9014f9b41b3f7a397f62da06e6ec4664b8b"}
Jul 03 09:48:26.802450 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:26.802338    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc" event={"ID":"47d7cde5-4688-4610-abd4-de8ff576fd75","Type":"ContainerStarted","Data":"41fe528d56351963bc2f7946f68b321ce31b4215a725237cc1f1102cd92c9ffc"}
Jul 03 09:48:26.802450 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:26.802374    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc" event={"ID":"47d7cde5-4688-4610-abd4-de8ff576fd75","Type":"ContainerStarted","Data":"4f7826d1411a871ac37baa58a7eb989f9a03353fe7a06c3c0afaf4400b958221"}
Jul 03 09:48:26.804649 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:26.804632    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg"
Jul 03 09:48:26.815915 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:26.815864    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-77pdg" podStartSLOduration=2.477247636 podStartE2EDuration="3.815846782s" totalImagesPullingTime="1.338599146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:48:26.814572612 +0000 UTC m=+308.679861280" watchObservedRunningTime="2026-07-03 09:48:26.815846782 +0000 UTC m=+308.681135433"
Jul 03 09:48:26.849728 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:26.849595    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc" podStartSLOduration=2.152320826 podStartE2EDuration="6.849575164s" totalImagesPullingTime="4.697254338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:48:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:48:26.848626265 +0000 UTC m=+308.713914949" watchObservedRunningTime="2026-07-03 09:48:26.849575164 +0000 UTC m=+308.714863820"
Jul 03 09:48:27.806555 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:27.806519    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:30.674974 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:30.674936    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:30.776182 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:30.776154    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:30.830995 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:30.830966    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:33.816319 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:33.816289    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-596fdd74fc-lgvlc"
Jul 03 09:48:34.091933 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:34.091852    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:34.189709 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:34.189677    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:34.290411 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:34.290382    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:34.391084 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:34.391013    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:34.491723 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:34.491689    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:34.591538 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:34.591491    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:34.693228 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:34.693192    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:34.792053 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:34.792001    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:35.039776 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:35.039674    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:35.139953 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:35.139924    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:35.358765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:35.358670    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:35.973682 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:35.973652    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:36.167305 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:36.167270    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:36.267940 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:36.267852    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:37.389365 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:37.389314    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:37.533446 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:37.533415    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:37.592500 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:37.592458    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:40.124742 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:40.124710    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:40.187200 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:40.187167    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:40.626415 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:40.626386    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:40.726113 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:40.726081    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:42.996263 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:42.996230    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:43.068508 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:43.068476    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:43.068664 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:43.068552    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:48:43.098216 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:43.098187    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:44.461690 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.461652    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bc98b9966-lq5gt"]
Jul 03 09:48:44.466000 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.465977    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.468444 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.468425    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 03 09:48:44.468547 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.468457    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 03 09:48:44.468547 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.468489    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 03 09:48:44.468547 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.468486    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 03 09:48:44.469503 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.469484    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 03 09:48:44.469613 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.469502    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 03 09:48:44.469613 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.469503    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-62tn6\""
Jul 03 09:48:44.469613 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.469502    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 03 09:48:44.474589 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.474571    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bc98b9966-lq5gt"]
Jul 03 09:48:44.505351 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.505324    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-oauth-config\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.505351 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.505353    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-config\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.505505 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.505371    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-oauth-serving-cert\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.505505 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.505433    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj5hz\" (UniqueName: \"kubernetes.io/projected/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-kube-api-access-zj5hz\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.505505 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.505463    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-serving-cert\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.505598 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.505529    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-service-ca\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.606082 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.606048    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-oauth-config\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.606082 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.606080    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-config\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.606308 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.606102    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-oauth-serving-cert\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.606308 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.606151    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj5hz\" (UniqueName: \"kubernetes.io/projected/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-kube-api-access-zj5hz\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.606308 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.606177    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-serving-cert\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.606308 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.606216    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-service-ca\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.606910 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.606888    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-oauth-serving-cert\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.607042 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.606909    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-config\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.607042 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.606950    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-service-ca\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.608535 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.608511    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-serving-cert\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.608622 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.608550    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-oauth-config\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.615284 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.615261    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj5hz\" (UniqueName: \"kubernetes.io/projected/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-kube-api-access-zj5hz\") pod \"console-bc98b9966-lq5gt\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") " pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.775409 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.775329    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:44.910522 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:44.910499    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bc98b9966-lq5gt"]
Jul 03 09:48:44.914222 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:48:44.914190    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a4f567_9138_4c8f_81d0_f7f837b2ea3a.slice/crio-72f7ff390c94c4893a9aa7dca015a19a2514546b65960d60cb781d8e1e5f8ab1 WatchSource:0}: Error finding container 72f7ff390c94c4893a9aa7dca015a19a2514546b65960d60cb781d8e1e5f8ab1: Status 404 returned error can't find the container with id 72f7ff390c94c4893a9aa7dca015a19a2514546b65960d60cb781d8e1e5f8ab1
Jul 03 09:48:45.750946 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:45.750908    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:45.813597 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:45.813549    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:45.872919 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:45.872875    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc98b9966-lq5gt" event={"ID":"18a4f567-9138-4c8f-81d0-f7f837b2ea3a","Type":"ContainerStarted","Data":"72f7ff390c94c4893a9aa7dca015a19a2514546b65960d60cb781d8e1e5f8ab1"}
Jul 03 09:48:46.622690 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:46.622630    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:46.722426 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:46.722388    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:48.886370 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:48.886339    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc98b9966-lq5gt" event={"ID":"18a4f567-9138-4c8f-81d0-f7f837b2ea3a","Type":"ContainerStarted","Data":"d0b5fd80203452384b8f556aaf52e17b72f83951f3ddc06833ec50be4dd6ff3c"}
Jul 03 09:48:48.904349 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:48.904297    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bc98b9966-lq5gt" podStartSLOduration=1.6274008370000002 podStartE2EDuration="4.904280671s" totalImagesPullingTime="3.276879834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:48:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:48:48.904090484 +0000 UTC m=+330.769379138" watchObservedRunningTime="2026-07-03 09:48:48.904280671 +0000 UTC m=+330.769569325"
Jul 03 09:48:49.384368 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:48:49.384334    2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bcdf3e0_7d30_4010_9219_8a8901e43f25.slice/crio-conmon-49516b387ce0b06fcba304e4224a8803dc13b08f353a28604f6b1e4bbead57b5.scope\": RecentStats: unable to find data in memory cache]"
Jul 03 09:48:49.601207 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:49.601120    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:49.892051 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:49.891969    2571 generic.go:356] "Generic (PLEG): container finished" podID="7bcdf3e0-7d30-4010-9219-8a8901e43f25" containerID="49516b387ce0b06fcba304e4224a8803dc13b08f353a28604f6b1e4bbead57b5" exitCode=0
Jul 03 09:48:49.892382 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:49.892045    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r" event={"ID":"7bcdf3e0-7d30-4010-9219-8a8901e43f25","Type":"ContainerDied","Data":"49516b387ce0b06fcba304e4224a8803dc13b08f353a28604f6b1e4bbead57b5"}
Jul 03 09:48:49.892459 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:49.892441    2571 scope.go:117] "RemoveContainer" containerID="49516b387ce0b06fcba304e4224a8803dc13b08f353a28604f6b1e4bbead57b5"
Jul 03 09:48:50.897899 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:50.897867    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wpk2r" event={"ID":"7bcdf3e0-7d30-4010-9219-8a8901e43f25","Type":"ContainerStarted","Data":"bae0037b8d847cb0c909cea554a99010a71b0c06bd4df3d70d542575ca410d7c"}
Jul 03 09:48:53.400878 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:53.400826    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:53.460429 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:53.460386    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:48:54.775714 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:54.775683    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:54.776128 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:54.775729    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:54.780301 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:54.780277    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:48:54.916006 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:48:54.915978    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:49:00.746487 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:00.746458    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:00.847062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:00.847032    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:02.206129 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:02.206095    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:02.402319 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:02.402283    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:02.463148 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:02.463065    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:03.073644 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:03.073615    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:49:03.077326 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:03.077305    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7966bfc5ff-8gnwj"
Jul 03 09:49:03.443628 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:03.443595    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bc98b9966-lq5gt"]
Jul 03 09:49:03.545558 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:03.545529    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:03.603800 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:03.603770    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:03.835570 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:03.835496    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:03.936909 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:03.936861    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:15.748367 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:15.748291    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:15.848061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:15.848034    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:28.465169 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.465124    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-bc98b9966-lq5gt" podUID="18a4f567-9138-4c8f-81d0-f7f837b2ea3a" containerName="console" containerID="cri-o://d0b5fd80203452384b8f556aaf52e17b72f83951f3ddc06833ec50be4dd6ff3c" gracePeriod=15
Jul 03 09:49:28.707535 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.707511    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:49:28.796758 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.796681    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-serving-cert\") pod \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") "
Jul 03 09:49:28.796758 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.796718    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zj5hz\" (UniqueName: \"kubernetes.io/projected/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-kube-api-access-zj5hz\") pod \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") "
Jul 03 09:49:28.797001 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.796760    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-service-ca\") pod \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") "
Jul 03 09:49:28.797001 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.796807    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-config\") pod \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") "
Jul 03 09:49:28.797001 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.796853    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-oauth-config\") pod \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") "
Jul 03 09:49:28.797001 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.796884    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-oauth-serving-cert\") pod \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\" (UID: \"18a4f567-9138-4c8f-81d0-f7f837b2ea3a\") "
Jul 03 09:49:28.797247 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.797220    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-service-ca" (OuterVolumeSpecName: "service-ca") pod "18a4f567-9138-4c8f-81d0-f7f837b2ea3a" (UID: "18a4f567-9138-4c8f-81d0-f7f837b2ea3a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 09:49:28.797247 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.797223    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-config" (OuterVolumeSpecName: "console-config") pod "18a4f567-9138-4c8f-81d0-f7f837b2ea3a" (UID: "18a4f567-9138-4c8f-81d0-f7f837b2ea3a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 09:49:28.797538 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.797517    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "18a4f567-9138-4c8f-81d0-f7f837b2ea3a" (UID: "18a4f567-9138-4c8f-81d0-f7f837b2ea3a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 09:49:28.799099 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.799064    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "18a4f567-9138-4c8f-81d0-f7f837b2ea3a" (UID: "18a4f567-9138-4c8f-81d0-f7f837b2ea3a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 09:49:28.799186 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.799077    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "18a4f567-9138-4c8f-81d0-f7f837b2ea3a" (UID: "18a4f567-9138-4c8f-81d0-f7f837b2ea3a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 09:49:28.799186 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.799119    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-kube-api-access-zj5hz" (OuterVolumeSpecName: "kube-api-access-zj5hz") pod "18a4f567-9138-4c8f-81d0-f7f837b2ea3a" (UID: "18a4f567-9138-4c8f-81d0-f7f837b2ea3a"). InnerVolumeSpecName "kube-api-access-zj5hz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:49:28.897853 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.897802    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-config\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:49:28.897853 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.897851    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-oauth-config\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:49:28.897853 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.897862    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-oauth-serving-cert\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:49:28.898085 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.897871    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-console-serving-cert\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:49:28.898085 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.897880    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zj5hz\" (UniqueName: \"kubernetes.io/projected/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-kube-api-access-zj5hz\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:49:28.898085 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:28.897890    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18a4f567-9138-4c8f-81d0-f7f837b2ea3a-service-ca\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:49:29.037535 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:29.037501    2571 generic.go:356] "Generic (PLEG): container finished" podID="18a4f567-9138-4c8f-81d0-f7f837b2ea3a" containerID="d0b5fd80203452384b8f556aaf52e17b72f83951f3ddc06833ec50be4dd6ff3c" exitCode=0
Jul 03 09:49:29.037688 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:29.037542    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc98b9966-lq5gt"
Jul 03 09:49:29.037688 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:29.037584    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc98b9966-lq5gt" event={"ID":"18a4f567-9138-4c8f-81d0-f7f837b2ea3a","Type":"ContainerDied","Data":"d0b5fd80203452384b8f556aaf52e17b72f83951f3ddc06833ec50be4dd6ff3c"}
Jul 03 09:49:29.037688 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:29.037637    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc98b9966-lq5gt" event={"ID":"18a4f567-9138-4c8f-81d0-f7f837b2ea3a","Type":"ContainerDied","Data":"72f7ff390c94c4893a9aa7dca015a19a2514546b65960d60cb781d8e1e5f8ab1"}
Jul 03 09:49:29.037688 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:29.037657    2571 scope.go:117] "RemoveContainer" containerID="d0b5fd80203452384b8f556aaf52e17b72f83951f3ddc06833ec50be4dd6ff3c"
Jul 03 09:49:29.048522 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:29.047940    2571 scope.go:117] "RemoveContainer" containerID="d0b5fd80203452384b8f556aaf52e17b72f83951f3ddc06833ec50be4dd6ff3c"
Jul 03 09:49:29.048930 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:49:29.048899    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0b5fd80203452384b8f556aaf52e17b72f83951f3ddc06833ec50be4dd6ff3c\": container with ID starting with d0b5fd80203452384b8f556aaf52e17b72f83951f3ddc06833ec50be4dd6ff3c not found: ID does not exist" containerID="d0b5fd80203452384b8f556aaf52e17b72f83951f3ddc06833ec50be4dd6ff3c"
Jul 03 09:49:29.049024 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:29.048933    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0b5fd80203452384b8f556aaf52e17b72f83951f3ddc06833ec50be4dd6ff3c"} err="failed to get container status \"d0b5fd80203452384b8f556aaf52e17b72f83951f3ddc06833ec50be4dd6ff3c\": rpc error: code = NotFound desc = could not find container \"d0b5fd80203452384b8f556aaf52e17b72f83951f3ddc06833ec50be4dd6ff3c\": container with ID starting with d0b5fd80203452384b8f556aaf52e17b72f83951f3ddc06833ec50be4dd6ff3c not found: ID does not exist"
Jul 03 09:49:29.058494 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:29.058469    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bc98b9966-lq5gt"]
Jul 03 09:49:29.063966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:29.063947    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bc98b9966-lq5gt"]
Jul 03 09:49:30.763821 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:30.763790    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18a4f567-9138-4c8f-81d0-f7f837b2ea3a" path="/var/lib/kubelet/pods/18a4f567-9138-4c8f-81d0-f7f837b2ea3a/volumes"
Jul 03 09:49:31.521986 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:31.521925    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:49:31.524270 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:31.524246    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f18ac135-e44e-413a-9bda-248eacb6287b-metrics-certs\") pod \"network-metrics-daemon-txhn2\" (UID: \"f18ac135-e44e-413a-9bda-248eacb6287b\") " pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:49:31.562337 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:31.562313    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-98cbg\""
Jul 03 09:49:31.569526 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:31.569506    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txhn2"
Jul 03 09:49:31.697386 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:31.697364    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-txhn2"]
Jul 03 09:49:31.699410 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:49:31.699383    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf18ac135_e44e_413a_9bda_248eacb6287b.slice/crio-84a2d5163e4fdee4475ce62e60240005a79fb2c1a8482212bf2924f958857096 WatchSource:0}: Error finding container 84a2d5163e4fdee4475ce62e60240005a79fb2c1a8482212bf2924f958857096: Status 404 returned error can't find the container with id 84a2d5163e4fdee4475ce62e60240005a79fb2c1a8482212bf2924f958857096
Jul 03 09:49:32.051619 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:32.051572    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-txhn2" event={"ID":"f18ac135-e44e-413a-9bda-248eacb6287b","Type":"ContainerStarted","Data":"84a2d5163e4fdee4475ce62e60240005a79fb2c1a8482212bf2924f958857096"}
Jul 03 09:49:32.630337 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:32.630306    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:32.730733 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:32.730707    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:33.057234 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:33.057183    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-txhn2" event={"ID":"f18ac135-e44e-413a-9bda-248eacb6287b","Type":"ContainerStarted","Data":"3bada269f8463b75385c87cff4c7678d50edb35497fe4d27c92293e729919b1a"}
Jul 03 09:49:33.057607 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:33.057261    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-txhn2" event={"ID":"f18ac135-e44e-413a-9bda-248eacb6287b","Type":"ContainerStarted","Data":"e34f049d36bd25a47236c94a26412c1ca9b44c60c1227d3b61f0648979bdf9c8"}
Jul 03 09:49:33.075206 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:33.075150    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-txhn2" podStartSLOduration=373.210392281 podStartE2EDuration="6m14.075131356s" totalImagesPullingTime="864.739075ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:49:33.074655341 +0000 UTC m=+374.939943998" watchObservedRunningTime="2026-07-03 09:49:33.075131356 +0000 UTC m=+374.940420014"
Jul 03 09:49:33.182184 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:33.182137    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:33.240035 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:33.240003    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:34.544559 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:34.544526    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:34.631645 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:34.631613    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:35.012854 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:35.012796    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:35.090312 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:35.090281    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:46.475640 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:46.475607    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:46.536477 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:46.536440    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:49.518303 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:49.518264    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:51.408649 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:51.408609    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:51.508643 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:51.508603    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:53.777050 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.777017    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-764c6c8597-qcgk8"]
Jul 03 09:49:53.777456 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.777309    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a4f567-9138-4c8f-81d0-f7f837b2ea3a" containerName="console"
Jul 03 09:49:53.777456 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.777319    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a4f567-9138-4c8f-81d0-f7f837b2ea3a" containerName="console"
Jul 03 09:49:53.777456 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.777373    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="18a4f567-9138-4c8f-81d0-f7f837b2ea3a" containerName="console"
Jul 03 09:49:53.780241 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.780217    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:53.782525 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.782501    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 03 09:49:53.783478 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.783457    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 03 09:49:53.783617 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.783593    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 03 09:49:53.783705 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.783618    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 03 09:49:53.783705 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.783521    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 03 09:49:53.783705 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.783571    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 03 09:49:53.783705 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.783546    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 03 09:49:53.784014 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.783458    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-62tn6\""
Jul 03 09:49:53.787900 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.787875    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 03 09:49:53.792925 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.792902    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-764c6c8597-qcgk8"]
Jul 03 09:49:53.836494 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.836455    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:53.907467 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.907429    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw45r\" (UniqueName: \"kubernetes.io/projected/7ce6f509-c01e-40df-b63b-51bced4bdb52-kube-api-access-mw45r\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:53.907467 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.907467    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-serving-cert\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:53.907725 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.907514    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-config\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:53.907725 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.907645    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-trusted-ca-bundle\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:53.907725 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.907676    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-service-ca\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:53.907903 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.907790    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-oauth-config\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:53.907959 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.907914    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-oauth-serving-cert\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:53.985343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:53.985297    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:54.008652 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.008626    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-trusted-ca-bundle\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.008760 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.008656    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-service-ca\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.008760 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.008680    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-oauth-config\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.008760 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.008710    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-oauth-serving-cert\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.008760 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.008729    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw45r\" (UniqueName: \"kubernetes.io/projected/7ce6f509-c01e-40df-b63b-51bced4bdb52-kube-api-access-mw45r\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.008942 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.008872    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-serving-cert\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.009042 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.008969    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-config\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.009575 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.009547    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-service-ca\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.009704 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.009590    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-oauth-serving-cert\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.009704 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.009609    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-trusted-ca-bundle\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.009704 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.009658    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-config\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.011197 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.011169    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-oauth-config\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.011387 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.011370    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-serving-cert\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.018172 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.018150    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw45r\" (UniqueName: \"kubernetes.io/projected/7ce6f509-c01e-40df-b63b-51bced4bdb52-kube-api-access-mw45r\") pod \"console-764c6c8597-qcgk8\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") " pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.044126 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.044062    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:54.090204 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.090160    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:49:54.220950 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.220917    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-764c6c8597-qcgk8"]
Jul 03 09:49:54.223427 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:49:54.223394    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce6f509_c01e_40df_b63b_51bced4bdb52.slice/crio-5845bf67e348d49cf48401a3a44f8427c3a6e1c6a2d1a8b8cff9ef486453cbf2 WatchSource:0}: Error finding container 5845bf67e348d49cf48401a3a44f8427c3a6e1c6a2d1a8b8cff9ef486453cbf2: Status 404 returned error can't find the container with id 5845bf67e348d49cf48401a3a44f8427c3a6e1c6a2d1a8b8cff9ef486453cbf2
Jul 03 09:49:54.575484 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.575445    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:54.634754 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:54.634726    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:49:55.142088 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:55.142043    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-764c6c8597-qcgk8" event={"ID":"7ce6f509-c01e-40df-b63b-51bced4bdb52","Type":"ContainerStarted","Data":"65ff00ef2365097c934fa9f8eab3a9f1aaa83e1f34cc2101f82b572370ef4afb"}
Jul 03 09:49:55.142487 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:55.142105    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-764c6c8597-qcgk8" event={"ID":"7ce6f509-c01e-40df-b63b-51bced4bdb52","Type":"ContainerStarted","Data":"5845bf67e348d49cf48401a3a44f8427c3a6e1c6a2d1a8b8cff9ef486453cbf2"}
Jul 03 09:49:55.161533 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:49:55.161486    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-764c6c8597-qcgk8" podStartSLOduration=2.161471385 podStartE2EDuration="2.161471385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:49:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:49:55.160481217 +0000 UTC m=+397.025769873" watchObservedRunningTime="2026-07-03 09:49:55.161471385 +0000 UTC m=+397.026760040"
Jul 03 09:50:04.090583 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:50:04.090549    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:50:04.091084 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:50:04.090868    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:50:04.095728 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:50:04.095697    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:50:04.179966 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:50:04.179937    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:50:04.330677 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:50:04.330630    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:50:04.490572 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:50:04.490543    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:50:04.546943 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:50:04.546910    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:50:37.294236 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:50:37.294202    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:50:42.612947 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:50:42.612820    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:50:42.673931 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:50:42.673894    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:00.964497 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:00.964466    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:01.063592 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:01.063560    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:28.118993 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:28.118957    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:28.960612 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:28.960581    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:31.996411 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:31.996350    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:32.097142 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:32.097104    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:33.466906 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:33.466870    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:33.648383 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:33.648339    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:33.973290 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:33.973255    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:34.106109 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:34.106072    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:36.332856 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.332806    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:36.565521 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.565491    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:36.727900 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.727861    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hss8j"]
Jul 03 09:51:36.731047 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.731030    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hss8j"
Jul 03 09:51:36.733619 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.733596    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 03 09:51:36.734530 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.734512    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mx5ww\""
Jul 03 09:51:36.734592 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.734512    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 03 09:51:36.740809 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.740787    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hss8j"]
Jul 03 09:51:36.859845 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.859806    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05cef0a7-6cc4-4b27-85dc-0861ec66a527-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hss8j\" (UID: \"05cef0a7-6cc4-4b27-85dc-0861ec66a527\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hss8j"
Jul 03 09:51:36.860037 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.859918    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbwmd\" (UniqueName: \"kubernetes.io/projected/05cef0a7-6cc4-4b27-85dc-0861ec66a527-kube-api-access-vbwmd\") pod \"cert-manager-cainjector-559cdd8c87-hss8j\" (UID: \"05cef0a7-6cc4-4b27-85dc-0861ec66a527\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hss8j"
Jul 03 09:51:36.960355 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.960317    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05cef0a7-6cc4-4b27-85dc-0861ec66a527-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hss8j\" (UID: \"05cef0a7-6cc4-4b27-85dc-0861ec66a527\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hss8j"
Jul 03 09:51:36.960489 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.960377    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbwmd\" (UniqueName: \"kubernetes.io/projected/05cef0a7-6cc4-4b27-85dc-0861ec66a527-kube-api-access-vbwmd\") pod \"cert-manager-cainjector-559cdd8c87-hss8j\" (UID: \"05cef0a7-6cc4-4b27-85dc-0861ec66a527\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hss8j"
Jul 03 09:51:36.970339 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.970299    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05cef0a7-6cc4-4b27-85dc-0861ec66a527-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hss8j\" (UID: \"05cef0a7-6cc4-4b27-85dc-0861ec66a527\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hss8j"
Jul 03 09:51:36.970572 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:36.970552    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbwmd\" (UniqueName: \"kubernetes.io/projected/05cef0a7-6cc4-4b27-85dc-0861ec66a527-kube-api-access-vbwmd\") pod \"cert-manager-cainjector-559cdd8c87-hss8j\" (UID: \"05cef0a7-6cc4-4b27-85dc-0861ec66a527\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hss8j"
Jul 03 09:51:37.046510 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:37.046425    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hss8j"
Jul 03 09:51:37.196190 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:37.196164    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hss8j"]
Jul 03 09:51:37.198258 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:51:37.198231    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05cef0a7_6cc4_4b27_85dc_0861ec66a527.slice/crio-d0ceda5fd2dd97cef95b1e3bda2f326f29ec7f3d6ccb6a213299508c9566598d WatchSource:0}: Error finding container d0ceda5fd2dd97cef95b1e3bda2f326f29ec7f3d6ccb6a213299508c9566598d: Status 404 returned error can't find the container with id d0ceda5fd2dd97cef95b1e3bda2f326f29ec7f3d6ccb6a213299508c9566598d
Jul 03 09:51:37.526737 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:37.526698    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hss8j" event={"ID":"05cef0a7-6cc4-4b27-85dc-0861ec66a527","Type":"ContainerStarted","Data":"d0ceda5fd2dd97cef95b1e3bda2f326f29ec7f3d6ccb6a213299508c9566598d"}
Jul 03 09:51:38.706739 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:38.706706    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:38.764806 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:38.764771    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:43.987536 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:43.987504    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:44.331554 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:44.331465    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:44.430764 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:44.430735    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:44.866063 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:44.866028    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9x5sf"]
Jul 03 09:51:44.869693 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:44.869667    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9x5sf"
Jul 03 09:51:44.872545 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:44.872225    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-cg6xp\""
Jul 03 09:51:44.879794 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:44.879253    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9x5sf"]
Jul 03 09:51:45.032948 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:45.032910    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2e32f80-c731-4bfe-9892-97b8a8b88cf9-bound-sa-token\") pod \"cert-manager-55b4d954d5-9x5sf\" (UID: \"f2e32f80-c731-4bfe-9892-97b8a8b88cf9\") " pod="cert-manager/cert-manager-55b4d954d5-9x5sf"
Jul 03 09:51:45.032948 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:45.032959    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdg9k\" (UniqueName: \"kubernetes.io/projected/f2e32f80-c731-4bfe-9892-97b8a8b88cf9-kube-api-access-sdg9k\") pod \"cert-manager-55b4d954d5-9x5sf\" (UID: \"f2e32f80-c731-4bfe-9892-97b8a8b88cf9\") " pod="cert-manager/cert-manager-55b4d954d5-9x5sf"
Jul 03 09:51:45.133964 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:45.133878    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2e32f80-c731-4bfe-9892-97b8a8b88cf9-bound-sa-token\") pod \"cert-manager-55b4d954d5-9x5sf\" (UID: \"f2e32f80-c731-4bfe-9892-97b8a8b88cf9\") " pod="cert-manager/cert-manager-55b4d954d5-9x5sf"
Jul 03 09:51:45.133964 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:45.133933    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdg9k\" (UniqueName: \"kubernetes.io/projected/f2e32f80-c731-4bfe-9892-97b8a8b88cf9-kube-api-access-sdg9k\") pod \"cert-manager-55b4d954d5-9x5sf\" (UID: \"f2e32f80-c731-4bfe-9892-97b8a8b88cf9\") " pod="cert-manager/cert-manager-55b4d954d5-9x5sf"
Jul 03 09:51:45.144706 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:45.144643    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2e32f80-c731-4bfe-9892-97b8a8b88cf9-bound-sa-token\") pod \"cert-manager-55b4d954d5-9x5sf\" (UID: \"f2e32f80-c731-4bfe-9892-97b8a8b88cf9\") " pod="cert-manager/cert-manager-55b4d954d5-9x5sf"
Jul 03 09:51:45.145394 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:45.145348    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdg9k\" (UniqueName: \"kubernetes.io/projected/f2e32f80-c731-4bfe-9892-97b8a8b88cf9-kube-api-access-sdg9k\") pod \"cert-manager-55b4d954d5-9x5sf\" (UID: \"f2e32f80-c731-4bfe-9892-97b8a8b88cf9\") " pod="cert-manager/cert-manager-55b4d954d5-9x5sf"
Jul 03 09:51:45.183204 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:45.183162    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9x5sf"
Jul 03 09:51:45.349425 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:45.349231    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9x5sf"]
Jul 03 09:51:45.352764 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:51:45.352119    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2e32f80_c731_4bfe_9892_97b8a8b88cf9.slice/crio-1b5b788421fbe4cd0cb95abecd10eb5776fdfb8994562276913309a138b0d3c0 WatchSource:0}: Error finding container 1b5b788421fbe4cd0cb95abecd10eb5776fdfb8994562276913309a138b0d3c0: Status 404 returned error can't find the container with id 1b5b788421fbe4cd0cb95abecd10eb5776fdfb8994562276913309a138b0d3c0
Jul 03 09:51:45.559485 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:45.559447    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9x5sf" event={"ID":"f2e32f80-c731-4bfe-9892-97b8a8b88cf9","Type":"ContainerStarted","Data":"1b5b788421fbe4cd0cb95abecd10eb5776fdfb8994562276913309a138b0d3c0"}
Jul 03 09:51:47.570919 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:47.570800    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hss8j" event={"ID":"05cef0a7-6cc4-4b27-85dc-0861ec66a527","Type":"ContainerStarted","Data":"bbaaa47cf61a228a98c3916be043cebb1c38c54f0bbaacab584a98cb8198ca09"}
Jul 03 09:51:47.572320 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:47.572297    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9x5sf" event={"ID":"f2e32f80-c731-4bfe-9892-97b8a8b88cf9","Type":"ContainerStarted","Data":"3040ee38188a831cb41562ccbdefd88f2e8b13121a86c710547ba0f5920e194e"}
Jul 03 09:51:47.588608 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:47.588559    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hss8j" podStartSLOduration=1.613892931 podStartE2EDuration="11.58854387s" totalImagesPullingTime="9.974650939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:51:47.588126754 +0000 UTC m=+509.453415423" watchObservedRunningTime="2026-07-03 09:51:47.58854387 +0000 UTC m=+509.453832524"
Jul 03 09:51:47.607004 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:47.606948    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9x5sf" podStartSLOduration=1.7855051020000001 podStartE2EDuration="3.606930944s" totalImagesPullingTime="1.821425842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:51:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:51:47.605593101 +0000 UTC m=+509.470881757" watchObservedRunningTime="2026-07-03 09:51:47.606930944 +0000 UTC m=+509.472219598"
Jul 03 09:51:47.985143 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:47.985100    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:48.055436 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:48.055405    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:50.788609 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:50.788579    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:50.890532 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:50.890500    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:52.452086 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:52.452046    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:51:56.409927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:51:56.409883    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:02.774369 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:02.774334    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:02.860076 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:02.860048    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:05.591018 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:05.590986    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:05.689849 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:05.689798    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:06.140546 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.140511    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:06.466234 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.466192    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:06.573471 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.573437    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:06.632029 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.631993    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"]
Jul 03 09:52:06.636469 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.636446    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.642355 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.642331    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 03 09:52:06.642808 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.642778    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 03 09:52:06.643455 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.642867    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zswg2\""
Jul 03 09:52:06.643455 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.643073    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 03 09:52:06.643455 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.643415    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 03 09:52:06.663189 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.663164    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"]
Jul 03 09:52:06.720561 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.720467    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-manifests\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.720561 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.720544    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-tmp\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.720751 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.720595    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.720751 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.720627    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2dr7\" (UniqueName: \"kubernetes.io/projected/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-kube-api-access-m2dr7\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.720751 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.720679    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.776807 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.776773    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:06.821691 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.821654    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-tmp\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.821892 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.821707    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.821892 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.821744    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2dr7\" (UniqueName: \"kubernetes.io/projected/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-kube-api-access-m2dr7\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.821892 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.821769    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.821892 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.821828    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-manifests\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.822209 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.822192    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-manifests\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.824153 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.824129    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-tmp\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.824330 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.824309    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.824369 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.824329    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.829896 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.829875    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2dr7\" (UniqueName: \"kubernetes.io/projected/a7cd9fda-9cec-4479-a0ac-779a5e0a38e0-kube-api-access-m2dr7\") pod \"opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6\" (UID: \"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0\") " pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.877383 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.877351    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:06.946429 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.946388    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:06.979712 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:06.979611    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:07.063347 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:07.063317    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:07.093910 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:07.093886    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"]
Jul 03 09:52:07.096148 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:52:07.096123    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7cd9fda_9cec_4479_a0ac_779a5e0a38e0.slice/crio-97d6a927ec1d2dac007bfb73a0bcf6df3e69f6113e43aab321cfdc4a7b047a0c WatchSource:0}: Error finding container 97d6a927ec1d2dac007bfb73a0bcf6df3e69f6113e43aab321cfdc4a7b047a0c: Status 404 returned error can't find the container with id 97d6a927ec1d2dac007bfb73a0bcf6df3e69f6113e43aab321cfdc4a7b047a0c
Jul 03 09:52:07.143755 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:07.143711    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:07.255145 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:07.255056    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:07.344731 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:07.344695    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:07.444016 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:07.443986    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:07.654005 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:07.653960    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6" event={"ID":"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0","Type":"ContainerStarted","Data":"97d6a927ec1d2dac007bfb73a0bcf6df3e69f6113e43aab321cfdc4a7b047a0c"}
Jul 03 09:52:10.670652 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:10.670609    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6" event={"ID":"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0","Type":"ContainerStarted","Data":"1d1fe28cb6be70ebf7d1098b465cfbae53d44a18ac88348610dac2ccfbde157b"}
Jul 03 09:52:11.676859 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:11.676807    2571 generic.go:356] "Generic (PLEG): container finished" podID="a7cd9fda-9cec-4479-a0ac-779a5e0a38e0" containerID="1d1fe28cb6be70ebf7d1098b465cfbae53d44a18ac88348610dac2ccfbde157b" exitCode=0
Jul 03 09:52:11.677361 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:11.676884    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6" event={"ID":"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0","Type":"ContainerDied","Data":"1d1fe28cb6be70ebf7d1098b465cfbae53d44a18ac88348610dac2ccfbde157b"}
Jul 03 09:52:12.683108 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:12.683073    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6" event={"ID":"a7cd9fda-9cec-4479-a0ac-779a5e0a38e0","Type":"ContainerStarted","Data":"8d7e3e10301f76da96cfe174d062fbbf4a346caceffdb83c684b92e48f886293"}
Jul 03 09:52:12.683496 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:12.683211    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:12.712887 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:12.712819    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6" podStartSLOduration=2.45158563 podStartE2EDuration="6.712803931s" totalImagesPullingTime="3.21040696s" totalInitContainerRuntime="1.050811341s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:52:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:52:12.711329322 +0000 UTC m=+534.576617999" watchObservedRunningTime="2026-07-03 09:52:12.712803931 +0000 UTC m=+534.578092587"
Jul 03 09:52:15.906351 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:15.906305    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:16.698811 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:16.698765    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:16.813089 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:16.813059    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:17.706653 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:17.706614    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:17.794758 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:17.794726    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:23.690007 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:23.689972    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6"
Jul 03 09:52:25.344242 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:25.344197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:25.563897 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:25.563861    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:25.982627 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:25.982591    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:26.060821 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.060783    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"]
Jul 03 09:52:26.073995 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.073966    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"
Jul 03 09:52:26.077246 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.077221    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 03 09:52:26.077350 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.077246    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gjsm8\""
Jul 03 09:52:26.078062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.078043    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 03 09:52:26.087629 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.087603    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"]
Jul 03 09:52:26.196014 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.195978    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ed2ca98-af98-40cc-a837-12963a215d81-tls-certs\") pod \"kube-auth-proxy-5f5df5b9bf-hknrl\" (UID: \"0ed2ca98-af98-40cc-a837-12963a215d81\") " pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"
Jul 03 09:52:26.196216 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.196040    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtqj9\" (UniqueName: \"kubernetes.io/projected/0ed2ca98-af98-40cc-a837-12963a215d81-kube-api-access-vtqj9\") pod \"kube-auth-proxy-5f5df5b9bf-hknrl\" (UID: \"0ed2ca98-af98-40cc-a837-12963a215d81\") " pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"
Jul 03 09:52:26.196216 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.196157    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ed2ca98-af98-40cc-a837-12963a215d81-tmp\") pod \"kube-auth-proxy-5f5df5b9bf-hknrl\" (UID: \"0ed2ca98-af98-40cc-a837-12963a215d81\") " pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"
Jul 03 09:52:26.205842 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.205811    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:26.297556 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.297468    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ed2ca98-af98-40cc-a837-12963a215d81-tmp\") pod \"kube-auth-proxy-5f5df5b9bf-hknrl\" (UID: \"0ed2ca98-af98-40cc-a837-12963a215d81\") " pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"
Jul 03 09:52:26.297556 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.297512    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ed2ca98-af98-40cc-a837-12963a215d81-tls-certs\") pod \"kube-auth-proxy-5f5df5b9bf-hknrl\" (UID: \"0ed2ca98-af98-40cc-a837-12963a215d81\") " pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"
Jul 03 09:52:26.297556 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.297558    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtqj9\" (UniqueName: \"kubernetes.io/projected/0ed2ca98-af98-40cc-a837-12963a215d81-kube-api-access-vtqj9\") pod \"kube-auth-proxy-5f5df5b9bf-hknrl\" (UID: \"0ed2ca98-af98-40cc-a837-12963a215d81\") " pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"
Jul 03 09:52:26.299896 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.299861    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ed2ca98-af98-40cc-a837-12963a215d81-tmp\") pod \"kube-auth-proxy-5f5df5b9bf-hknrl\" (UID: \"0ed2ca98-af98-40cc-a837-12963a215d81\") " pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"
Jul 03 09:52:26.300021 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.299868    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ed2ca98-af98-40cc-a837-12963a215d81-tls-certs\") pod \"kube-auth-proxy-5f5df5b9bf-hknrl\" (UID: \"0ed2ca98-af98-40cc-a837-12963a215d81\") " pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"
Jul 03 09:52:26.320239 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.320203    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtqj9\" (UniqueName: \"kubernetes.io/projected/0ed2ca98-af98-40cc-a837-12963a215d81-kube-api-access-vtqj9\") pod \"kube-auth-proxy-5f5df5b9bf-hknrl\" (UID: \"0ed2ca98-af98-40cc-a837-12963a215d81\") " pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"
Jul 03 09:52:26.383308 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.383272    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"
Jul 03 09:52:26.425983 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.425949    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-p5rm4"]
Jul 03 09:52:26.438109 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.438080    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4"
Jul 03 09:52:26.440458 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.440310    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-p5rm4"]
Jul 03 09:52:26.441583 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.441559    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-bvfb8\""
Jul 03 09:52:26.442055 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.441862    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 03 09:52:26.499719 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.499680    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6llvm\" (UniqueName: \"kubernetes.io/projected/ef02fb74-8757-4753-a979-c926bcfb7b9f-kube-api-access-6llvm\") pod \"odh-model-controller-7998c67dc4-p5rm4\" (UID: \"ef02fb74-8757-4753-a979-c926bcfb7b9f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4"
Jul 03 09:52:26.499909 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.499744    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef02fb74-8757-4753-a979-c926bcfb7b9f-cert\") pod \"odh-model-controller-7998c67dc4-p5rm4\" (UID: \"ef02fb74-8757-4753-a979-c926bcfb7b9f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4"
Jul 03 09:52:26.509027 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.508996    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:26.521429 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.521352    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl"]
Jul 03 09:52:26.524219 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:52:26.524194    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed2ca98_af98_40cc_a837_12963a215d81.slice/crio-a1c3040b36f2d5827febea947cbd580b22aa8d499440c8ef980e13275c6c30b9 WatchSource:0}: Error finding container a1c3040b36f2d5827febea947cbd580b22aa8d499440c8ef980e13275c6c30b9: Status 404 returned error can't find the container with id a1c3040b36f2d5827febea947cbd580b22aa8d499440c8ef980e13275c6c30b9
Jul 03 09:52:26.600601 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.600518    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6llvm\" (UniqueName: \"kubernetes.io/projected/ef02fb74-8757-4753-a979-c926bcfb7b9f-kube-api-access-6llvm\") pod \"odh-model-controller-7998c67dc4-p5rm4\" (UID: \"ef02fb74-8757-4753-a979-c926bcfb7b9f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4"
Jul 03 09:52:26.600601 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.600558    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef02fb74-8757-4753-a979-c926bcfb7b9f-cert\") pod \"odh-model-controller-7998c67dc4-p5rm4\" (UID: \"ef02fb74-8757-4753-a979-c926bcfb7b9f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4"
Jul 03 09:52:26.600814 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:52:26.600662    2571 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 03 09:52:26.600814 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:52:26.600727    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef02fb74-8757-4753-a979-c926bcfb7b9f-cert podName:ef02fb74-8757-4753-a979-c926bcfb7b9f nodeName:}" failed. No retries permitted until 2026-07-03 09:52:27.100709097 +0000 UTC m=+548.965997732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ef02fb74-8757-4753-a979-c926bcfb7b9f-cert") pod "odh-model-controller-7998c67dc4-p5rm4" (UID: "ef02fb74-8757-4753-a979-c926bcfb7b9f") : secret "odh-model-controller-webhook-cert" not found
Jul 03 09:52:26.610934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.610903    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:26.614049 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.614024    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6llvm\" (UniqueName: \"kubernetes.io/projected/ef02fb74-8757-4753-a979-c926bcfb7b9f-kube-api-access-6llvm\") pod \"odh-model-controller-7998c67dc4-p5rm4\" (UID: \"ef02fb74-8757-4753-a979-c926bcfb7b9f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4"
Jul 03 09:52:26.708594 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.708558    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:26.745653 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.745620    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl" event={"ID":"0ed2ca98-af98-40cc-a837-12963a215d81","Type":"ContainerStarted","Data":"a1c3040b36f2d5827febea947cbd580b22aa8d499440c8ef980e13275c6c30b9"}
Jul 03 09:52:26.858080 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:26.857994    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:27.104311 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.104271    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef02fb74-8757-4753-a979-c926bcfb7b9f-cert\") pod \"odh-model-controller-7998c67dc4-p5rm4\" (UID: \"ef02fb74-8757-4753-a979-c926bcfb7b9f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4"
Jul 03 09:52:27.106786 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.106761    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef02fb74-8757-4753-a979-c926bcfb7b9f-cert\") pod \"odh-model-controller-7998c67dc4-p5rm4\" (UID: \"ef02fb74-8757-4753-a979-c926bcfb7b9f\") " pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4"
Jul 03 09:52:27.279054 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.278983    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:27.323609 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.323577    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mtgkq"]
Jul 03 09:52:27.326427 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.326399    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq"
Jul 03 09:52:27.330084 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.330062    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 03 09:52:27.340749 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.340726    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mtgkq"]
Jul 03 09:52:27.353718 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.353693    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4"
Jul 03 09:52:27.386416 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.386379    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:27.408334 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.408090    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1ae1da7-3d46-482e-9554-2cf34350243e-tls-certs\") pod \"model-serving-api-54d89bd5fc-mtgkq\" (UID: \"b1ae1da7-3d46-482e-9554-2cf34350243e\") " pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq"
Jul 03 09:52:27.408334 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.408164    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnlvh\" (UniqueName: \"kubernetes.io/projected/b1ae1da7-3d46-482e-9554-2cf34350243e-kube-api-access-xnlvh\") pod \"model-serving-api-54d89bd5fc-mtgkq\" (UID: \"b1ae1da7-3d46-482e-9554-2cf34350243e\") " pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq"
Jul 03 09:52:27.509593 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.509558    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnlvh\" (UniqueName: \"kubernetes.io/projected/b1ae1da7-3d46-482e-9554-2cf34350243e-kube-api-access-xnlvh\") pod \"model-serving-api-54d89bd5fc-mtgkq\" (UID: \"b1ae1da7-3d46-482e-9554-2cf34350243e\") " pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq"
Jul 03 09:52:27.509782 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.509637    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1ae1da7-3d46-482e-9554-2cf34350243e-tls-certs\") pod \"model-serving-api-54d89bd5fc-mtgkq\" (UID: \"b1ae1da7-3d46-482e-9554-2cf34350243e\") " pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq"
Jul 03 09:52:27.509871 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:52:27.509784    2571 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 03 09:52:27.509871 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:52:27.509858    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1ae1da7-3d46-482e-9554-2cf34350243e-tls-certs podName:b1ae1da7-3d46-482e-9554-2cf34350243e nodeName:}" failed. No retries permitted until 2026-07-03 09:52:28.009825112 +0000 UTC m=+549.875113755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b1ae1da7-3d46-482e-9554-2cf34350243e-tls-certs") pod "model-serving-api-54d89bd5fc-mtgkq" (UID: "b1ae1da7-3d46-482e-9554-2cf34350243e") : secret "model-serving-api-tls" not found
Jul 03 09:52:27.513862 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.513816    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-p5rm4"]
Jul 03 09:52:27.518125 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:52:27.518096    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef02fb74_8757_4753_a979_c926bcfb7b9f.slice/crio-3df373f908514fc4938d9e92c801200e1e209597a0758e860081570543fd0609 WatchSource:0}: Error finding container 3df373f908514fc4938d9e92c801200e1e209597a0758e860081570543fd0609: Status 404 returned error can't find the container with id 3df373f908514fc4938d9e92c801200e1e209597a0758e860081570543fd0609
Jul 03 09:52:27.521489 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.521178    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnlvh\" (UniqueName: \"kubernetes.io/projected/b1ae1da7-3d46-482e-9554-2cf34350243e-kube-api-access-xnlvh\") pod \"model-serving-api-54d89bd5fc-mtgkq\" (UID: \"b1ae1da7-3d46-482e-9554-2cf34350243e\") " pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq"
Jul 03 09:52:27.578096 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.577805    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"]
Jul 03 09:52:27.578500 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.578450    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:27.581851 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.581816    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.584915 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.584596    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 03 09:52:27.584915 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.584612    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 03 09:52:27.584915 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.584723    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 03 09:52:27.584915 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.584786    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 03 09:52:27.584915 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.584908    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-c92nq\""
Jul 03 09:52:27.599069 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.598721    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"]
Jul 03 09:52:27.695623 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.695584    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:27.712017 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.711982    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwlnb\" (UniqueName: \"kubernetes.io/projected/78e6e225-12e1-42c8-a764-7f6bbf8d7540-kube-api-access-kwlnb\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.712209 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.712088    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78e6e225-12e1-42c8-a764-7f6bbf8d7540-local-certs\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.712209 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.712145    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.712209 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.712181    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.712387 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.712229    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.712387 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.712256    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-token\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.712387 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.712308    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-cacerts\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.751460 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.751420    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4" event={"ID":"ef02fb74-8757-4753-a979-c926bcfb7b9f","Type":"ContainerStarted","Data":"3df373f908514fc4938d9e92c801200e1e209597a0758e860081570543fd0609"}
Jul 03 09:52:27.813814 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.813780    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwlnb\" (UniqueName: \"kubernetes.io/projected/78e6e225-12e1-42c8-a764-7f6bbf8d7540-kube-api-access-kwlnb\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.814042 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.813846    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78e6e225-12e1-42c8-a764-7f6bbf8d7540-local-certs\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.814042 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.814010    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.814178 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.814152    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.814239 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.814212    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.814297 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.814245    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-token\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.814350 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.814303    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-cacerts\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.814734 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.814707    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.816679 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.816655    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78e6e225-12e1-42c8-a764-7f6bbf8d7540-local-certs\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.817064 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.817039    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-cacerts\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.817192 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.817074    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.817255 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.817208    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.823526 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.823492    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-token\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.823783 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.823762    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwlnb\" (UniqueName: \"kubernetes.io/projected/78e6e225-12e1-42c8-a764-7f6bbf8d7540-kube-api-access-kwlnb\") pod \"istiod-openshift-gateway-7574459656-vrnzn\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:27.846722 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.846648    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:27.893478 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:27.893442    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:28.016324 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:28.016288    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1ae1da7-3d46-482e-9554-2cf34350243e-tls-certs\") pod \"model-serving-api-54d89bd5fc-mtgkq\" (UID: \"b1ae1da7-3d46-482e-9554-2cf34350243e\") " pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq"
Jul 03 09:52:28.018983 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:28.018960    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1ae1da7-3d46-482e-9554-2cf34350243e-tls-certs\") pod \"model-serving-api-54d89bd5fc-mtgkq\" (UID: \"b1ae1da7-3d46-482e-9554-2cf34350243e\") " pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq"
Jul 03 09:52:28.048657 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:28.048629    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"]
Jul 03 09:52:28.050009 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:52:28.049981    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78e6e225_12e1_42c8_a764_7f6bbf8d7540.slice/crio-64d01a0124ae8d7e4ce38677a2bb306e997eccb11b7b1be585bd02d9fb11d54b WatchSource:0}: Error finding container 64d01a0124ae8d7e4ce38677a2bb306e997eccb11b7b1be585bd02d9fb11d54b: Status 404 returned error can't find the container with id 64d01a0124ae8d7e4ce38677a2bb306e997eccb11b7b1be585bd02d9fb11d54b
Jul 03 09:52:28.240659 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:28.240357    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq"
Jul 03 09:52:28.433085 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:28.433043    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mtgkq"]
Jul 03 09:52:28.765022 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:28.764993    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn" event={"ID":"78e6e225-12e1-42c8-a764-7f6bbf8d7540","Type":"ContainerStarted","Data":"64d01a0124ae8d7e4ce38677a2bb306e997eccb11b7b1be585bd02d9fb11d54b"}
Jul 03 09:52:29.097759 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:29.097623    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:29.992139 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:52:29.992100    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ae1da7_3d46_482e_9554_2cf34350243e.slice/crio-aa2ffcc012a3f1714bc40ab29337c8b5d1b217338ee6d9ad94a5d3c166edd532 WatchSource:0}: Error finding container aa2ffcc012a3f1714bc40ab29337c8b5d1b217338ee6d9ad94a5d3c166edd532: Status 404 returned error can't find the container with id aa2ffcc012a3f1714bc40ab29337c8b5d1b217338ee6d9ad94a5d3c166edd532
Jul 03 09:52:30.770845 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:30.770799    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq" event={"ID":"b1ae1da7-3d46-482e-9554-2cf34350243e","Type":"ContainerStarted","Data":"aa2ffcc012a3f1714bc40ab29337c8b5d1b217338ee6d9ad94a5d3c166edd532"}
Jul 03 09:52:32.151998 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:32.151955    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 03 09:52:32.152385 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:32.152042    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 03 09:52:32.782563 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:32.782524    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq" event={"ID":"b1ae1da7-3d46-482e-9554-2cf34350243e","Type":"ContainerStarted","Data":"d42b152240b921f42ea8cda03596732840277c0998be03455c441792deae9299"}
Jul 03 09:52:32.782766 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:32.782590    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq"
Jul 03 09:52:32.784087 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:32.784054    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl" event={"ID":"0ed2ca98-af98-40cc-a837-12963a215d81","Type":"ContainerStarted","Data":"9317754ac869ff92d8f5c7771865bf99e7e86e01c4cfa4605913a39116bc686b"}
Jul 03 09:52:32.785548 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:32.785520    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4" event={"ID":"ef02fb74-8757-4753-a979-c926bcfb7b9f","Type":"ContainerStarted","Data":"11955a7df2f7ae8fc48f4d9ea50b34f3b119c1d5201752d5f1f20f0aea45a6e5"}
Jul 03 09:52:32.785687 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:32.785623    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4"
Jul 03 09:52:32.787132 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:32.787106    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn" event={"ID":"78e6e225-12e1-42c8-a764-7f6bbf8d7540","Type":"ContainerStarted","Data":"c12d7baec82e65da3c0d4f723c0b1c72d3265ac104ce3b1745798be49075b8f6"}
Jul 03 09:52:32.787267 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:32.787242    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:32.827222 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:32.827178    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq" podStartSLOduration=3.652276197 podStartE2EDuration="5.827164734s" totalImagesPullingTime="2.174888537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:52:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:52:32.820232267 +0000 UTC m=+554.685520923" watchObservedRunningTime="2026-07-03 09:52:32.827164734 +0000 UTC m=+554.692453385"
Jul 03 09:52:32.868812 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:32.868761    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4" podStartSLOduration=2.642117455 podStartE2EDuration="6.868745547s" totalImagesPullingTime="4.226628092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:52:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:52:32.865801748 +0000 UTC m=+554.731090403" watchObservedRunningTime="2026-07-03 09:52:32.868745547 +0000 UTC m=+554.734034201"
Jul 03 09:52:32.954787 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:32.954732    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5f5df5b9bf-hknrl" podStartSLOduration=1.7446750020000001 podStartE2EDuration="6.954716849s" totalImagesPullingTime="5.210041847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:52:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:52:32.952439867 +0000 UTC m=+554.817728513" watchObservedRunningTime="2026-07-03 09:52:32.954716849 +0000 UTC m=+554.820005504"
Jul 03 09:52:33.025020 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:33.024967    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn" podStartSLOduration=1.925601302 podStartE2EDuration="6.024950931s" totalImagesPullingTime="4.099349629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:52:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:52:33.022964955 +0000 UTC m=+554.888253601" watchObservedRunningTime="2026-07-03 09:52:33.024950931 +0000 UTC m=+554.890239586"
Jul 03 09:52:33.342872 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:33.342826    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:33.521598 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:33.521565    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:33.774165 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:33.774126    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:33.793983 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:33.793951    2571 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-vrnzn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 03 09:52:33.794133 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:33.794005    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn" podUID="78e6e225-12e1-42c8-a764-7f6bbf8d7540" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 03 09:52:33.953050 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:33.953018    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:34.269182 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:34.269144    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:36.793932 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:36.793897    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:39.564011 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:39.563979    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:39.850027 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:39.849938    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:40.027194 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.027158    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-llfzz"]
Jul 03 09:52:40.031119 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.031100    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-llfzz"
Jul 03 09:52:40.037878 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.037528    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 03 09:52:40.037878 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.037547    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4vqqr\""
Jul 03 09:52:40.069569 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.069534    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"]
Jul 03 09:52:40.072421 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.072404    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"
Jul 03 09:52:40.082423 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.082392    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-llfzz"]
Jul 03 09:52:40.084971 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.084948    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 03 09:52:40.085078 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.084992    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-5shf9\""
Jul 03 09:52:40.105203 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.105118    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:40.108074 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.108040    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"]
Jul 03 09:52:40.133279 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.133240    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7cc7\" (UniqueName: \"kubernetes.io/projected/482fd11c-ac13-482a-befc-ab34e0d98a21-kube-api-access-n7cc7\") pod \"kserve-controller-manager-d6f4b65b8-llfzz\" (UID: \"482fd11c-ac13-482a-befc-ab34e0d98a21\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-llfzz"
Jul 03 09:52:40.133446 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.133388    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/482fd11c-ac13-482a-befc-ab34e0d98a21-cert\") pod \"kserve-controller-manager-d6f4b65b8-llfzz\" (UID: \"482fd11c-ac13-482a-befc-ab34e0d98a21\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-llfzz"
Jul 03 09:52:40.234449 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.234414    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cee01c2-e4cf-4c25-8337-293f81074e2b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9xrx7\" (UID: \"0cee01c2-e4cf-4c25-8337-293f81074e2b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"
Jul 03 09:52:40.234449 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.234454    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/482fd11c-ac13-482a-befc-ab34e0d98a21-cert\") pod \"kserve-controller-manager-d6f4b65b8-llfzz\" (UID: \"482fd11c-ac13-482a-befc-ab34e0d98a21\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-llfzz"
Jul 03 09:52:40.234697 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.234479    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7cc7\" (UniqueName: \"kubernetes.io/projected/482fd11c-ac13-482a-befc-ab34e0d98a21-kube-api-access-n7cc7\") pod \"kserve-controller-manager-d6f4b65b8-llfzz\" (UID: \"482fd11c-ac13-482a-befc-ab34e0d98a21\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-llfzz"
Jul 03 09:52:40.234697 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.234500    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skbgh\" (UniqueName: \"kubernetes.io/projected/0cee01c2-e4cf-4c25-8337-293f81074e2b-kube-api-access-skbgh\") pod \"llmisvc-controller-manager-777c6bd7d5-9xrx7\" (UID: \"0cee01c2-e4cf-4c25-8337-293f81074e2b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"
Jul 03 09:52:40.236989 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.236969    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/482fd11c-ac13-482a-befc-ab34e0d98a21-cert\") pod \"kserve-controller-manager-d6f4b65b8-llfzz\" (UID: \"482fd11c-ac13-482a-befc-ab34e0d98a21\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-llfzz"
Jul 03 09:52:40.258248 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.258216    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7cc7\" (UniqueName: \"kubernetes.io/projected/482fd11c-ac13-482a-befc-ab34e0d98a21-kube-api-access-n7cc7\") pod \"kserve-controller-manager-d6f4b65b8-llfzz\" (UID: \"482fd11c-ac13-482a-befc-ab34e0d98a21\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-llfzz"
Jul 03 09:52:40.319496 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.319458    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:40.335899 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.335828    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cee01c2-e4cf-4c25-8337-293f81074e2b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9xrx7\" (UID: \"0cee01c2-e4cf-4c25-8337-293f81074e2b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"
Jul 03 09:52:40.336097 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.335930    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skbgh\" (UniqueName: \"kubernetes.io/projected/0cee01c2-e4cf-4c25-8337-293f81074e2b-kube-api-access-skbgh\") pod \"llmisvc-controller-manager-777c6bd7d5-9xrx7\" (UID: \"0cee01c2-e4cf-4c25-8337-293f81074e2b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"
Jul 03 09:52:40.336097 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:52:40.336003    2571 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 03 09:52:40.336097 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:52:40.336080    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cee01c2-e4cf-4c25-8337-293f81074e2b-cert podName:0cee01c2-e4cf-4c25-8337-293f81074e2b nodeName:}" failed. No retries permitted until 2026-07-03 09:52:40.836057272 +0000 UTC m=+562.701345911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0cee01c2-e4cf-4c25-8337-293f81074e2b-cert") pod "llmisvc-controller-manager-777c6bd7d5-9xrx7" (UID: "0cee01c2-e4cf-4c25-8337-293f81074e2b") : secret "llmisvc-webhook-server-cert" not found
Jul 03 09:52:40.341858 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.341819    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-llfzz"
Jul 03 09:52:40.353502 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.353468    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skbgh\" (UniqueName: \"kubernetes.io/projected/0cee01c2-e4cf-4c25-8337-293f81074e2b-kube-api-access-skbgh\") pod \"llmisvc-controller-manager-777c6bd7d5-9xrx7\" (UID: \"0cee01c2-e4cf-4c25-8337-293f81074e2b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"
Jul 03 09:52:40.421613 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.421557    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:40.511855 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.511796    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-llfzz"]
Jul 03 09:52:40.516792 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:52:40.516762    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod482fd11c_ac13_482a_befc_ab34e0d98a21.slice/crio-836585263f3e26658486fbbae55f180987de63093b165a2ed2b97098ec7e274a WatchSource:0}: Error finding container 836585263f3e26658486fbbae55f180987de63093b165a2ed2b97098ec7e274a: Status 404 returned error can't find the container with id 836585263f3e26658486fbbae55f180987de63093b165a2ed2b97098ec7e274a
Jul 03 09:52:40.519427 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.519402    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:52:40.825815 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.825779    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-llfzz" event={"ID":"482fd11c-ac13-482a-befc-ab34e0d98a21","Type":"ContainerStarted","Data":"836585263f3e26658486fbbae55f180987de63093b165a2ed2b97098ec7e274a"}
Jul 03 09:52:40.840359 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.840327    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cee01c2-e4cf-4c25-8337-293f81074e2b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9xrx7\" (UID: \"0cee01c2-e4cf-4c25-8337-293f81074e2b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"
Jul 03 09:52:40.842695 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.842674    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0cee01c2-e4cf-4c25-8337-293f81074e2b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9xrx7\" (UID: \"0cee01c2-e4cf-4c25-8337-293f81074e2b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"
Jul 03 09:52:40.983311 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:40.983270    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"
Jul 03 09:52:41.143946 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:41.143872    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"]
Jul 03 09:52:41.146700 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:52:41.146665    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0cee01c2_e4cf_4c25_8337_293f81074e2b.slice/crio-47a9a3849d23a7ae9d8eb0eb121e8c6b809dcc8f364f9a7ce40beb124321bdfa WatchSource:0}: Error finding container 47a9a3849d23a7ae9d8eb0eb121e8c6b809dcc8f364f9a7ce40beb124321bdfa: Status 404 returned error can't find the container with id 47a9a3849d23a7ae9d8eb0eb121e8c6b809dcc8f364f9a7ce40beb124321bdfa
Jul 03 09:52:41.834392 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:41.834347    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7" event={"ID":"0cee01c2-e4cf-4c25-8337-293f81074e2b","Type":"ContainerStarted","Data":"47a9a3849d23a7ae9d8eb0eb121e8c6b809dcc8f364f9a7ce40beb124321bdfa"}
Jul 03 09:52:43.795941 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:43.795889    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-p5rm4"
Jul 03 09:52:43.797679 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:43.797656    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-mtgkq"
Jul 03 09:52:43.849015 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:43.848931    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-llfzz" event={"ID":"482fd11c-ac13-482a-befc-ab34e0d98a21","Type":"ContainerStarted","Data":"7a860bf973aeb07353edac01654172670dfdf6018536b5fe7b1ce6a608eed01e"}
Jul 03 09:52:43.849202 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:43.849127    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-llfzz"
Jul 03 09:52:43.875797 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:43.875730    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-llfzz" podStartSLOduration=2.3495348 podStartE2EDuration="4.875712871s" totalImagesPullingTime="2.526178071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:52:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:52:43.874380942 +0000 UTC m=+565.739669596" watchObservedRunningTime="2026-07-03 09:52:43.875712871 +0000 UTC m=+565.741001563"
Jul 03 09:52:44.855558 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:44.855519    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7" event={"ID":"0cee01c2-e4cf-4c25-8337-293f81074e2b","Type":"ContainerStarted","Data":"aac9dbe7f0d89b803dea0bb21d292d42f9c300a7c63b2ccc632bc57cfbf014ef"}
Jul 03 09:52:44.856025 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:44.855739    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"
Jul 03 09:52:44.887131 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:44.887077    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7" podStartSLOduration=2.654602043 podStartE2EDuration="5.887062685s" totalImagesPullingTime="3.232460642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:52:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:52:44.885397146 +0000 UTC m=+566.750685803" watchObservedRunningTime="2026-07-03 09:52:44.887062685 +0000 UTC m=+566.752351340"
Jul 03 09:52:54.389619 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.389585    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"]
Jul 03 09:52:54.390130 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.389785    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn" podUID="78e6e225-12e1-42c8-a764-7f6bbf8d7540" containerName="discovery" containerID="cri-o://c12d7baec82e65da3c0d4f723c0b1c72d3265ac104ce3b1745798be49075b8f6" gracePeriod=30
Jul 03 09:52:54.638596 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.638568    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:54.769230 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.769193    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwlnb\" (UniqueName: \"kubernetes.io/projected/78e6e225-12e1-42c8-a764-7f6bbf8d7540-kube-api-access-kwlnb\") pod \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") "
Jul 03 09:52:54.769381 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.769259    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78e6e225-12e1-42c8-a764-7f6bbf8d7540-local-certs\") pod \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") "
Jul 03 09:52:54.769381 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.769358    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-csr-dns-cert\") pod \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") "
Jul 03 09:52:54.769486 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.769391    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-token\") pod \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") "
Jul 03 09:52:54.769486 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.769444    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-cacerts\") pod \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") "
Jul 03 09:52:54.769598 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.769498    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-kubeconfig\") pod \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") "
Jul 03 09:52:54.769598 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.769548    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-csr-ca-configmap\") pod \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\" (UID: \"78e6e225-12e1-42c8-a764-7f6bbf8d7540\") "
Jul 03 09:52:54.770262 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.770217    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "78e6e225-12e1-42c8-a764-7f6bbf8d7540" (UID: "78e6e225-12e1-42c8-a764-7f6bbf8d7540"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 09:52:54.771874 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.771800    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78e6e225-12e1-42c8-a764-7f6bbf8d7540-local-certs" (OuterVolumeSpecName: "local-certs") pod "78e6e225-12e1-42c8-a764-7f6bbf8d7540" (UID: "78e6e225-12e1-42c8-a764-7f6bbf8d7540"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 09:52:54.771980 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.771939    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "78e6e225-12e1-42c8-a764-7f6bbf8d7540" (UID: "78e6e225-12e1-42c8-a764-7f6bbf8d7540"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 09:52:54.771980 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.771965    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78e6e225-12e1-42c8-a764-7f6bbf8d7540-kube-api-access-kwlnb" (OuterVolumeSpecName: "kube-api-access-kwlnb") pod "78e6e225-12e1-42c8-a764-7f6bbf8d7540" (UID: "78e6e225-12e1-42c8-a764-7f6bbf8d7540"). InnerVolumeSpecName "kube-api-access-kwlnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:52:54.772188 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.772072    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-token" (OuterVolumeSpecName: "istio-token") pod "78e6e225-12e1-42c8-a764-7f6bbf8d7540" (UID: "78e6e225-12e1-42c8-a764-7f6bbf8d7540"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:52:54.772188 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.772180    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "78e6e225-12e1-42c8-a764-7f6bbf8d7540" (UID: "78e6e225-12e1-42c8-a764-7f6bbf8d7540"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 09:52:54.772339 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.772314    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-cacerts" (OuterVolumeSpecName: "cacerts") pod "78e6e225-12e1-42c8-a764-7f6bbf8d7540" (UID: "78e6e225-12e1-42c8-a764-7f6bbf8d7540"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 09:52:54.871239 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.871201    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwlnb\" (UniqueName: \"kubernetes.io/projected/78e6e225-12e1-42c8-a764-7f6bbf8d7540-kube-api-access-kwlnb\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:52:54.871239 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.871237    2571 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78e6e225-12e1-42c8-a764-7f6bbf8d7540-local-certs\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:52:54.871439 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.871249    2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-csr-dns-cert\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:52:54.871439 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.871258    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-token\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:52:54.871439 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.871266    2571 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-cacerts\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:52:54.871439 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.871274    2571 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-kubeconfig\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:52:54.871439 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.871281    2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78e6e225-12e1-42c8-a764-7f6bbf8d7540-istio-csr-ca-configmap\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:52:54.902276 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.902243    2571 generic.go:356] "Generic (PLEG): container finished" podID="78e6e225-12e1-42c8-a764-7f6bbf8d7540" containerID="c12d7baec82e65da3c0d4f723c0b1c72d3265ac104ce3b1745798be49075b8f6" exitCode=0
Jul 03 09:52:54.902431 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.902296    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"
Jul 03 09:52:54.902431 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.902336    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn" event={"ID":"78e6e225-12e1-42c8-a764-7f6bbf8d7540","Type":"ContainerDied","Data":"c12d7baec82e65da3c0d4f723c0b1c72d3265ac104ce3b1745798be49075b8f6"}
Jul 03 09:52:54.902431 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.902389    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn" event={"ID":"78e6e225-12e1-42c8-a764-7f6bbf8d7540","Type":"ContainerDied","Data":"64d01a0124ae8d7e4ce38677a2bb306e997eccb11b7b1be585bd02d9fb11d54b"}
Jul 03 09:52:54.902431 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.902410    2571 scope.go:117] "RemoveContainer" containerID="c12d7baec82e65da3c0d4f723c0b1c72d3265ac104ce3b1745798be49075b8f6"
Jul 03 09:52:54.911490 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.911473    2571 scope.go:117] "RemoveContainer" containerID="c12d7baec82e65da3c0d4f723c0b1c72d3265ac104ce3b1745798be49075b8f6"
Jul 03 09:52:54.911772 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:52:54.911752    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c12d7baec82e65da3c0d4f723c0b1c72d3265ac104ce3b1745798be49075b8f6\": container with ID starting with c12d7baec82e65da3c0d4f723c0b1c72d3265ac104ce3b1745798be49075b8f6 not found: ID does not exist" containerID="c12d7baec82e65da3c0d4f723c0b1c72d3265ac104ce3b1745798be49075b8f6"
Jul 03 09:52:54.911863 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.911786    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c12d7baec82e65da3c0d4f723c0b1c72d3265ac104ce3b1745798be49075b8f6"} err="failed to get container status \"c12d7baec82e65da3c0d4f723c0b1c72d3265ac104ce3b1745798be49075b8f6\": rpc error: code = NotFound desc = could not find container \"c12d7baec82e65da3c0d4f723c0b1c72d3265ac104ce3b1745798be49075b8f6\": container with ID starting with c12d7baec82e65da3c0d4f723c0b1c72d3265ac104ce3b1745798be49075b8f6 not found: ID does not exist"
Jul 03 09:52:54.941415 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.941373    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"]
Jul 03 09:52:54.947242 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:54.947217    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vrnzn"]
Jul 03 09:52:56.765431 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:52:56.765387    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78e6e225-12e1-42c8-a764-7f6bbf8d7540" path="/var/lib/kubelet/pods/78e6e225-12e1-42c8-a764-7f6bbf8d7540/volumes"
Jul 03 09:53:14.860801 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:14.860771    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-llfzz"
Jul 03 09:53:15.862920 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:15.862889    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9xrx7"
Jul 03 09:53:18.625771 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:18.625746    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 09:53:18.626758 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:18.626742    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 09:53:25.957089 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:25.957051    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:53:27.076453 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.076420    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h5ccb"]
Jul 03 09:53:27.076817 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.076755    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e6e225-12e1-42c8-a764-7f6bbf8d7540" containerName="discovery"
Jul 03 09:53:27.076817 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.076767    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e6e225-12e1-42c8-a764-7f6bbf8d7540" containerName="discovery"
Jul 03 09:53:27.076817 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.076816    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="78e6e225-12e1-42c8-a764-7f6bbf8d7540" containerName="discovery"
Jul 03 09:53:27.079533 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.079502    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h5ccb"
Jul 03 09:53:27.082112 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.082095    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 03 09:53:27.082186 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.082109    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 03 09:53:27.082186 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.082103    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-jv522\""
Jul 03 09:53:27.089367 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.088326    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h5ccb"]
Jul 03 09:53:27.147644 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.147607    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8rl2\" (UniqueName: \"kubernetes.io/projected/a3777889-963a-4857-b184-c18382a592fd-kube-api-access-l8rl2\") pod \"kuadrant-operator-catalog-h5ccb\" (UID: \"a3777889-963a-4857-b184-c18382a592fd\") " pod="kuadrant-system/kuadrant-operator-catalog-h5ccb"
Jul 03 09:53:27.181586 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.181551    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:53:27.249149 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.249110    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8rl2\" (UniqueName: \"kubernetes.io/projected/a3777889-963a-4857-b184-c18382a592fd-kube-api-access-l8rl2\") pod \"kuadrant-operator-catalog-h5ccb\" (UID: \"a3777889-963a-4857-b184-c18382a592fd\") " pod="kuadrant-system/kuadrant-operator-catalog-h5ccb"
Jul 03 09:53:27.258882 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.258827    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8rl2\" (UniqueName: \"kubernetes.io/projected/a3777889-963a-4857-b184-c18382a592fd-kube-api-access-l8rl2\") pod \"kuadrant-operator-catalog-h5ccb\" (UID: \"a3777889-963a-4857-b184-c18382a592fd\") " pod="kuadrant-system/kuadrant-operator-catalog-h5ccb"
Jul 03 09:53:27.390752 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.390670    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h5ccb"
Jul 03 09:53:27.449566 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.449538    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h5ccb"]
Jul 03 09:53:27.522556 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.522527    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h5ccb"]
Jul 03 09:53:27.524621 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:53:27.524600    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3777889_963a_4857_b184_c18382a592fd.slice/crio-3e955b47940797dd7aa64a2a013e87a10f8df21816ef07b52a46bec123680789 WatchSource:0}: Error finding container 3e955b47940797dd7aa64a2a013e87a10f8df21816ef07b52a46bec123680789: Status 404 returned error can't find the container with id 3e955b47940797dd7aa64a2a013e87a10f8df21816ef07b52a46bec123680789
Jul 03 09:53:27.525992 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.525974    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 09:53:27.657902 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.657870    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6xtc7"]
Jul 03 09:53:27.662191 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.662170    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6xtc7"
Jul 03 09:53:27.672316 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.672291    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6xtc7"]
Jul 03 09:53:27.752987 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.752947    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhdg6\" (UniqueName: \"kubernetes.io/projected/42019489-0140-4d95-b62d-7ee8a0e1abc2-kube-api-access-bhdg6\") pod \"kuadrant-operator-catalog-6xtc7\" (UID: \"42019489-0140-4d95-b62d-7ee8a0e1abc2\") " pod="kuadrant-system/kuadrant-operator-catalog-6xtc7"
Jul 03 09:53:27.853570 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.853528    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhdg6\" (UniqueName: \"kubernetes.io/projected/42019489-0140-4d95-b62d-7ee8a0e1abc2-kube-api-access-bhdg6\") pod \"kuadrant-operator-catalog-6xtc7\" (UID: \"42019489-0140-4d95-b62d-7ee8a0e1abc2\") " pod="kuadrant-system/kuadrant-operator-catalog-6xtc7"
Jul 03 09:53:27.863158 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.863127    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhdg6\" (UniqueName: \"kubernetes.io/projected/42019489-0140-4d95-b62d-7ee8a0e1abc2-kube-api-access-bhdg6\") pod \"kuadrant-operator-catalog-6xtc7\" (UID: \"42019489-0140-4d95-b62d-7ee8a0e1abc2\") " pod="kuadrant-system/kuadrant-operator-catalog-6xtc7"
Jul 03 09:53:27.877817 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.877789    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:53:27.972002 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:27.971915    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6xtc7"
Jul 03 09:53:28.060241 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:28.060190    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h5ccb" event={"ID":"a3777889-963a-4857-b184-c18382a592fd","Type":"ContainerStarted","Data":"3e955b47940797dd7aa64a2a013e87a10f8df21816ef07b52a46bec123680789"}
Jul 03 09:53:28.082788 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:28.082748    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:53:28.109707 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:28.109677    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6xtc7"]
Jul 03 09:53:28.112695 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:53:28.112673    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42019489_0140_4d95_b62d_7ee8a0e1abc2.slice/crio-c1cf05eae12aefc7f043f39785b190ce5ae0aa89a657cee3c58d3c92a58480a0 WatchSource:0}: Error finding container c1cf05eae12aefc7f043f39785b190ce5ae0aa89a657cee3c58d3c92a58480a0: Status 404 returned error can't find the container with id c1cf05eae12aefc7f043f39785b190ce5ae0aa89a657cee3c58d3c92a58480a0
Jul 03 09:53:29.068810 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:29.068775    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6xtc7" event={"ID":"42019489-0140-4d95-b62d-7ee8a0e1abc2","Type":"ContainerStarted","Data":"c1cf05eae12aefc7f043f39785b190ce5ae0aa89a657cee3c58d3c92a58480a0"}
Jul 03 09:53:30.074340 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:30.074306    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h5ccb" event={"ID":"a3777889-963a-4857-b184-c18382a592fd","Type":"ContainerStarted","Data":"0ceb48774141b2509839827cff81965a30ddadcb14f033056da3e60bdd506b65"}
Jul 03 09:53:30.074802 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:30.074367    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-h5ccb" podUID="a3777889-963a-4857-b184-c18382a592fd" containerName="registry-server" containerID="cri-o://0ceb48774141b2509839827cff81965a30ddadcb14f033056da3e60bdd506b65" gracePeriod=2
Jul 03 09:53:30.076000 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:30.075976    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6xtc7" event={"ID":"42019489-0140-4d95-b62d-7ee8a0e1abc2","Type":"ContainerStarted","Data":"0fe36ae960ff13869f8d70846c71c6888e502ed46cd8d1865ab8131f9cb24490"}
Jul 03 09:53:30.091116 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:30.091075    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h5ccb" podStartSLOduration=1.040015089 podStartE2EDuration="3.091060739s" totalImagesPullingTime="2.05104565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:53:30.089512713 +0000 UTC m=+611.954801370" watchObservedRunningTime="2026-07-03 09:53:30.091060739 +0000 UTC m=+611.956349393"
Jul 03 09:53:30.106240 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:30.106194    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6xtc7" podStartSLOduration=1.642515535 podStartE2EDuration="3.106180477s" totalImagesPullingTime="1.463664942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:53:30.104362792 +0000 UTC m=+611.969651450" watchObservedRunningTime="2026-07-03 09:53:30.106180477 +0000 UTC m=+611.971469131"
Jul 03 09:53:30.319563 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:30.319539    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h5ccb"
Jul 03 09:53:30.374277 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:30.374195    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8rl2\" (UniqueName: \"kubernetes.io/projected/a3777889-963a-4857-b184-c18382a592fd-kube-api-access-l8rl2\") pod \"a3777889-963a-4857-b184-c18382a592fd\" (UID: \"a3777889-963a-4857-b184-c18382a592fd\") "
Jul 03 09:53:30.376323 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:30.376293    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3777889-963a-4857-b184-c18382a592fd-kube-api-access-l8rl2" (OuterVolumeSpecName: "kube-api-access-l8rl2") pod "a3777889-963a-4857-b184-c18382a592fd" (UID: "a3777889-963a-4857-b184-c18382a592fd"). InnerVolumeSpecName "kube-api-access-l8rl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:53:30.475013 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:30.474979    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8rl2\" (UniqueName: \"kubernetes.io/projected/a3777889-963a-4857-b184-c18382a592fd-kube-api-access-l8rl2\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:53:31.082511 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:31.082418    2571 generic.go:356] "Generic (PLEG): container finished" podID="a3777889-963a-4857-b184-c18382a592fd" containerID="0ceb48774141b2509839827cff81965a30ddadcb14f033056da3e60bdd506b65" exitCode=0
Jul 03 09:53:31.082511 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:31.082460    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h5ccb"
Jul 03 09:53:31.083020 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:31.082506    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h5ccb" event={"ID":"a3777889-963a-4857-b184-c18382a592fd","Type":"ContainerDied","Data":"0ceb48774141b2509839827cff81965a30ddadcb14f033056da3e60bdd506b65"}
Jul 03 09:53:31.083020 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:31.082556    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h5ccb" event={"ID":"a3777889-963a-4857-b184-c18382a592fd","Type":"ContainerDied","Data":"3e955b47940797dd7aa64a2a013e87a10f8df21816ef07b52a46bec123680789"}
Jul 03 09:53:31.083020 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:31.082583    2571 scope.go:117] "RemoveContainer" containerID="0ceb48774141b2509839827cff81965a30ddadcb14f033056da3e60bdd506b65"
Jul 03 09:53:31.091825 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:31.091805    2571 scope.go:117] "RemoveContainer" containerID="0ceb48774141b2509839827cff81965a30ddadcb14f033056da3e60bdd506b65"
Jul 03 09:53:31.092129 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:53:31.092109    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ceb48774141b2509839827cff81965a30ddadcb14f033056da3e60bdd506b65\": container with ID starting with 0ceb48774141b2509839827cff81965a30ddadcb14f033056da3e60bdd506b65 not found: ID does not exist" containerID="0ceb48774141b2509839827cff81965a30ddadcb14f033056da3e60bdd506b65"
Jul 03 09:53:31.092206 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:31.092140    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ceb48774141b2509839827cff81965a30ddadcb14f033056da3e60bdd506b65"} err="failed to get container status \"0ceb48774141b2509839827cff81965a30ddadcb14f033056da3e60bdd506b65\": rpc error: code = NotFound desc = could not find container \"0ceb48774141b2509839827cff81965a30ddadcb14f033056da3e60bdd506b65\": container with ID starting with 0ceb48774141b2509839827cff81965a30ddadcb14f033056da3e60bdd506b65 not found: ID does not exist"
Jul 03 09:53:31.106682 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:31.106659    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h5ccb"]
Jul 03 09:53:31.108792 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:31.108770    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h5ccb"]
Jul 03 09:53:32.764635 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:32.764601    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3777889-963a-4857-b184-c18382a592fd" path="/var/lib/kubelet/pods/a3777889-963a-4857-b184-c18382a592fd/volumes"
Jul 03 09:53:37.972270 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:37.972234    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6xtc7"
Jul 03 09:53:37.972760 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:37.972291    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6xtc7"
Jul 03 09:53:37.994508 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:37.994485    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6xtc7"
Jul 03 09:53:38.137133 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:38.137107    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6xtc7"
Jul 03 09:53:49.662244 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.662212    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-784df47f47-jmxnc"]
Jul 03 09:53:49.662736 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.662551    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3777889-963a-4857-b184-c18382a592fd" containerName="registry-server"
Jul 03 09:53:49.662736 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.662563    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3777889-963a-4857-b184-c18382a592fd" containerName="registry-server"
Jul 03 09:53:49.662736 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.662629    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3777889-963a-4857-b184-c18382a592fd" containerName="registry-server"
Jul 03 09:53:49.667876 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.667852    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.684810 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.684782    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-784df47f47-jmxnc"]
Jul 03 09:53:49.736945 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.736908    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/488ce27b-8d7a-495f-84be-e568e33eb263-console-oauth-config\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.737114 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.736956    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/488ce27b-8d7a-495f-84be-e568e33eb263-console-serving-cert\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.737114 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.736981    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb52q\" (UniqueName: \"kubernetes.io/projected/488ce27b-8d7a-495f-84be-e568e33eb263-kube-api-access-nb52q\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.737114 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.737022    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/488ce27b-8d7a-495f-84be-e568e33eb263-oauth-serving-cert\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.737114 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.737041    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/488ce27b-8d7a-495f-84be-e568e33eb263-trusted-ca-bundle\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.737114 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.737059    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/488ce27b-8d7a-495f-84be-e568e33eb263-service-ca\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.737292 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.737133    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/488ce27b-8d7a-495f-84be-e568e33eb263-console-config\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.758249 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.758206    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:53:49.837651 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.837610    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/488ce27b-8d7a-495f-84be-e568e33eb263-oauth-serving-cert\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.837651 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.837650    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/488ce27b-8d7a-495f-84be-e568e33eb263-trusted-ca-bundle\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.837922 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.837668    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/488ce27b-8d7a-495f-84be-e568e33eb263-service-ca\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.837922 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.837751    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/488ce27b-8d7a-495f-84be-e568e33eb263-console-config\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.837922 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.837790    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/488ce27b-8d7a-495f-84be-e568e33eb263-console-oauth-config\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.837922 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.837879    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/488ce27b-8d7a-495f-84be-e568e33eb263-console-serving-cert\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.837922 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.837908    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb52q\" (UniqueName: \"kubernetes.io/projected/488ce27b-8d7a-495f-84be-e568e33eb263-kube-api-access-nb52q\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.838512 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.838483    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/488ce27b-8d7a-495f-84be-e568e33eb263-oauth-serving-cert\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.838609 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.838528    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/488ce27b-8d7a-495f-84be-e568e33eb263-trusted-ca-bundle\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.838674 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.838645    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/488ce27b-8d7a-495f-84be-e568e33eb263-console-config\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.838827 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.838808    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/488ce27b-8d7a-495f-84be-e568e33eb263-service-ca\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.840372 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.840350    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/488ce27b-8d7a-495f-84be-e568e33eb263-console-serving-cert\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.840476 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.840403    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/488ce27b-8d7a-495f-84be-e568e33eb263-console-oauth-config\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.849791 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.849765    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb52q\" (UniqueName: \"kubernetes.io/projected/488ce27b-8d7a-495f-84be-e568e33eb263-kube-api-access-nb52q\") pod \"console-784df47f47-jmxnc\" (UID: \"488ce27b-8d7a-495f-84be-e568e33eb263\") " pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:49.858504 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.858469    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:53:49.976791 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:49.976707    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:50.121926 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:50.121894    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-784df47f47-jmxnc"]
Jul 03 09:53:50.125601 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:53:50.125578    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod488ce27b_8d7a_495f_84be_e568e33eb263.slice/crio-75fb87c1919b02c37e4ab1a0621e715b44e0434f5f8d7755f418febce7a0bc55 WatchSource:0}: Error finding container 75fb87c1919b02c37e4ab1a0621e715b44e0434f5f8d7755f418febce7a0bc55: Status 404 returned error can't find the container with id 75fb87c1919b02c37e4ab1a0621e715b44e0434f5f8d7755f418febce7a0bc55
Jul 03 09:53:50.171405 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:50.171195    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784df47f47-jmxnc" event={"ID":"488ce27b-8d7a-495f-84be-e568e33eb263","Type":"ContainerStarted","Data":"75fb87c1919b02c37e4ab1a0621e715b44e0434f5f8d7755f418febce7a0bc55"}
Jul 03 09:53:51.177325 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:51.177287    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784df47f47-jmxnc" event={"ID":"488ce27b-8d7a-495f-84be-e568e33eb263","Type":"ContainerStarted","Data":"b105c3e673417e3c042ebc4b157e43233a9dfae454b8b9f201133b4aa3225e86"}
Jul 03 09:53:51.202329 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:51.202280    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-784df47f47-jmxnc" podStartSLOduration=2.202265104 podStartE2EDuration="2.202265104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:53:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:53:51.200464646 +0000 UTC m=+633.065753337" watchObservedRunningTime="2026-07-03 09:53:51.202265104 +0000 UTC m=+633.067553758"
Jul 03 09:53:56.017601 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:56.017566    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:53:56.619577 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:56.619538    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:53:57.672325 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:57.672285    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:53:57.765497 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:57.765466    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:53:59.977192 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:59.977139    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:59.977643 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:59.977212    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:53:59.982318 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:53:59.982293    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:54:00.102862 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.102814    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6wng7"]
Jul 03 09:54:00.106333 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.106316    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6wng7"
Jul 03 09:54:00.109075 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.109056    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-j7hbr\""
Jul 03 09:54:00.119429 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.119404    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6wng7"]
Jul 03 09:54:00.222577 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.222549    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-784df47f47-jmxnc"
Jul 03 09:54:00.238450 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.238391    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5898n\" (UniqueName: \"kubernetes.io/projected/b1d9c001-990a-4aae-abd0-807b117ddd46-kube-api-access-5898n\") pod \"authorino-operator-75cc8d5cb7-6wng7\" (UID: \"b1d9c001-990a-4aae-abd0-807b117ddd46\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6wng7"
Jul 03 09:54:00.339291 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.339258    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5898n\" (UniqueName: \"kubernetes.io/projected/b1d9c001-990a-4aae-abd0-807b117ddd46-kube-api-access-5898n\") pod \"authorino-operator-75cc8d5cb7-6wng7\" (UID: \"b1d9c001-990a-4aae-abd0-807b117ddd46\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6wng7"
Jul 03 09:54:00.360419 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.360384    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5898n\" (UniqueName: \"kubernetes.io/projected/b1d9c001-990a-4aae-abd0-807b117ddd46-kube-api-access-5898n\") pod \"authorino-operator-75cc8d5cb7-6wng7\" (UID: \"b1d9c001-990a-4aae-abd0-807b117ddd46\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6wng7"
Jul 03 09:54:00.366703 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.366673    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-764c6c8597-qcgk8"]
Jul 03 09:54:00.416956 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.416921    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6wng7"
Jul 03 09:54:00.564795 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.564772    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6wng7"]
Jul 03 09:54:00.567099 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:54:00.567070    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d9c001_990a_4aae_abd0_807b117ddd46.slice/crio-cf2c0b0b885703a7a2489d922cff064850ec25f32f000470bb56141c2b824c47 WatchSource:0}: Error finding container cf2c0b0b885703a7a2489d922cff064850ec25f32f000470bb56141c2b824c47: Status 404 returned error can't find the container with id cf2c0b0b885703a7a2489d922cff064850ec25f32f000470bb56141c2b824c47
Jul 03 09:54:00.610750 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.610715    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:00.709626 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.709585    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:00.811579 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.811496    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:00.922185 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:00.922152    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:01.239640 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:01.239532    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6wng7" event={"ID":"b1d9c001-990a-4aae-abd0-807b117ddd46","Type":"ContainerStarted","Data":"cf2c0b0b885703a7a2489d922cff064850ec25f32f000470bb56141c2b824c47"}
Jul 03 09:54:03.250653 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:03.250613    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6wng7" event={"ID":"b1d9c001-990a-4aae-abd0-807b117ddd46","Type":"ContainerStarted","Data":"3097417050ceecab2d15d6bc1f8e980cc9beeac6ed52b63915f6c5199d5190e5"}
Jul 03 09:54:03.251082 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:03.250707    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6wng7"
Jul 03 09:54:03.285012 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:03.284954    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6wng7" podStartSLOduration=1.105070681 podStartE2EDuration="3.284938627s" totalImagesPullingTime="2.179867946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:54:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:54:03.283292253 +0000 UTC m=+645.148580907" watchObservedRunningTime="2026-07-03 09:54:03.284938627 +0000 UTC m=+645.150227282"
Jul 03 09:54:04.048153 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.048115    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"]
Jul 03 09:54:04.051642 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.051623    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:04.056850 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.056803    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-blwqx\""
Jul 03 09:54:04.075598 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.075572    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"]
Jul 03 09:54:04.174927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.174891    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkzl2\" (UniqueName: \"kubernetes.io/projected/640c6872-5609-4aa6-b3bf-213fda077942-kube-api-access-qkzl2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" (UID: \"640c6872-5609-4aa6-b3bf-213fda077942\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:04.175072 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.174943    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/640c6872-5609-4aa6-b3bf-213fda077942-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" (UID: \"640c6872-5609-4aa6-b3bf-213fda077942\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:04.257684 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.257638    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:04.276376 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.276346    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkzl2\" (UniqueName: \"kubernetes.io/projected/640c6872-5609-4aa6-b3bf-213fda077942-kube-api-access-qkzl2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" (UID: \"640c6872-5609-4aa6-b3bf-213fda077942\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:04.276524 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.276392    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/640c6872-5609-4aa6-b3bf-213fda077942-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" (UID: \"640c6872-5609-4aa6-b3bf-213fda077942\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:04.276845 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.276816    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/640c6872-5609-4aa6-b3bf-213fda077942-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" (UID: \"640c6872-5609-4aa6-b3bf-213fda077942\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:04.292025 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.292000    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkzl2\" (UniqueName: \"kubernetes.io/projected/640c6872-5609-4aa6-b3bf-213fda077942-kube-api-access-qkzl2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" (UID: \"640c6872-5609-4aa6-b3bf-213fda077942\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:04.361643 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.361552    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:04.510806 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.510771    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"]
Jul 03 09:54:04.514037 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:54:04.514012    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod640c6872_5609_4aa6_b3bf_213fda077942.slice/crio-9a489e8f74b5cd93ddfbc37ffc5da5e42dcbce823d3a8bfffb9199e1990570cc WatchSource:0}: Error finding container 9a489e8f74b5cd93ddfbc37ffc5da5e42dcbce823d3a8bfffb9199e1990570cc: Status 404 returned error can't find the container with id 9a489e8f74b5cd93ddfbc37ffc5da5e42dcbce823d3a8bfffb9199e1990570cc
Jul 03 09:54:04.883420 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:04.883385    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:05.263084 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:05.263044    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj" event={"ID":"640c6872-5609-4aa6-b3bf-213fda077942","Type":"ContainerStarted","Data":"9a489e8f74b5cd93ddfbc37ffc5da5e42dcbce823d3a8bfffb9199e1990570cc"}
Jul 03 09:54:06.005302 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:06.005263    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"]
Jul 03 09:54:06.009117 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:06.009090    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:06.011628 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:06.011606    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hzs4j\""
Jul 03 09:54:06.027667 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:06.027636    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"]
Jul 03 09:54:06.094226 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:06.094187    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwsz7\" (UniqueName: \"kubernetes.io/projected/bb21412f-51de-4857-a799-0fcbf11f585e-kube-api-access-gwsz7\") pod \"limitador-operator-controller-manager-859d7b78bd-zb6x7\" (UID: \"bb21412f-51de-4857-a799-0fcbf11f585e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:06.195033 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:06.194992    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwsz7\" (UniqueName: \"kubernetes.io/projected/bb21412f-51de-4857-a799-0fcbf11f585e-kube-api-access-gwsz7\") pod \"limitador-operator-controller-manager-859d7b78bd-zb6x7\" (UID: \"bb21412f-51de-4857-a799-0fcbf11f585e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:06.203490 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:06.203462    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwsz7\" (UniqueName: \"kubernetes.io/projected/bb21412f-51de-4857-a799-0fcbf11f585e-kube-api-access-gwsz7\") pod \"limitador-operator-controller-manager-859d7b78bd-zb6x7\" (UID: \"bb21412f-51de-4857-a799-0fcbf11f585e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:06.322459 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:06.322369    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:06.431714 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:06.431504    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:06.476326 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:06.476186    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"]
Jul 03 09:54:06.479151 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:54:06.479126    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb21412f_51de_4857_a799_0fcbf11f585e.slice/crio-4581a9f75f1cd42a7665f4abfaa7be204e4d4a6c55f94e99b8b8df96d487632e WatchSource:0}: Error finding container 4581a9f75f1cd42a7665f4abfaa7be204e4d4a6c55f94e99b8b8df96d487632e: Status 404 returned error can't find the container with id 4581a9f75f1cd42a7665f4abfaa7be204e4d4a6c55f94e99b8b8df96d487632e
Jul 03 09:54:07.058353 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:07.058322    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:07.279516 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:07.279472    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7" event={"ID":"bb21412f-51de-4857-a799-0fcbf11f585e","Type":"ContainerStarted","Data":"4581a9f75f1cd42a7665f4abfaa7be204e4d4a6c55f94e99b8b8df96d487632e"}
Jul 03 09:54:08.099677 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:08.099639    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7qhwb"]
Jul 03 09:54:08.103825 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:08.103800    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7qhwb"
Jul 03 09:54:08.109781 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:08.109741    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 03 09:54:08.109952 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:08.109741    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5zzcp\""
Jul 03 09:54:08.123002 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:08.122926    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7qhwb"]
Jul 03 09:54:08.213577 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:08.213530    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97l8d\" (UniqueName: \"kubernetes.io/projected/a200bf34-d3ed-409b-8104-4ca1384c07a0-kube-api-access-97l8d\") pod \"dns-operator-controller-manager-864dd78c4c-7qhwb\" (UID: \"a200bf34-d3ed-409b-8104-4ca1384c07a0\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7qhwb"
Jul 03 09:54:08.314604 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:08.314565    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97l8d\" (UniqueName: \"kubernetes.io/projected/a200bf34-d3ed-409b-8104-4ca1384c07a0-kube-api-access-97l8d\") pod \"dns-operator-controller-manager-864dd78c4c-7qhwb\" (UID: \"a200bf34-d3ed-409b-8104-4ca1384c07a0\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7qhwb"
Jul 03 09:54:08.327111 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:08.327076    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97l8d\" (UniqueName: \"kubernetes.io/projected/a200bf34-d3ed-409b-8104-4ca1384c07a0-kube-api-access-97l8d\") pod \"dns-operator-controller-manager-864dd78c4c-7qhwb\" (UID: \"a200bf34-d3ed-409b-8104-4ca1384c07a0\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7qhwb"
Jul 03 09:54:08.418617 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:08.418574    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7qhwb"
Jul 03 09:54:09.435785 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:09.435753    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:09.539460 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:09.539433    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:09.644309 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:09.644276    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7qhwb"]
Jul 03 09:54:09.646451 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:54:09.646422    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda200bf34_d3ed_409b_8104_4ca1384c07a0.slice/crio-66309c1114cb26631db53c74a71ea938c08471c1cf44bd2307ccb2396007e3fb WatchSource:0}: Error finding container 66309c1114cb26631db53c74a71ea938c08471c1cf44bd2307ccb2396007e3fb: Status 404 returned error can't find the container with id 66309c1114cb26631db53c74a71ea938c08471c1cf44bd2307ccb2396007e3fb
Jul 03 09:54:09.834532 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:09.834453    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:09.936520 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:09.936486    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:10.296208 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:10.296168    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj" event={"ID":"640c6872-5609-4aa6-b3bf-213fda077942","Type":"ContainerStarted","Data":"f6c12d915f1b3156e0d2cba83d19794fda49be0b0f137ce95cbccf6900b5ced7"}
Jul 03 09:54:10.296396 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:10.296337    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:10.297593 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:10.297561    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7qhwb" event={"ID":"a200bf34-d3ed-409b-8104-4ca1384c07a0","Type":"ContainerStarted","Data":"66309c1114cb26631db53c74a71ea938c08471c1cf44bd2307ccb2396007e3fb"}
Jul 03 09:54:10.299311 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:10.299279    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7" event={"ID":"bb21412f-51de-4857-a799-0fcbf11f585e","Type":"ContainerStarted","Data":"0e4fce182e923e02e282c584b1009dba920317f82cab30e21050db9e7be96e00"}
Jul 03 09:54:10.299416 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:10.299400    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:10.352868 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:10.352798    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj" podStartSLOduration=1.358803669 podStartE2EDuration="6.352780729s" totalImagesPullingTime="4.99397706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:54:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:54:10.351531114 +0000 UTC m=+652.216819768" watchObservedRunningTime="2026-07-03 09:54:10.352780729 +0000 UTC m=+652.218069385"
Jul 03 09:54:10.388259 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:10.388194    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7" podStartSLOduration=2.372387881 podStartE2EDuration="5.388173783s" totalImagesPullingTime="3.015785902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:54:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:54:10.37840783 +0000 UTC m=+652.243696484" watchObservedRunningTime="2026-07-03 09:54:10.388173783 +0000 UTC m=+652.253462439"
Jul 03 09:54:12.314414 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.314379    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7qhwb" event={"ID":"a200bf34-d3ed-409b-8104-4ca1384c07a0","Type":"ContainerStarted","Data":"b067190cd1b9aba79fe4c33175ec88576a8be8fc0347460d682080ac6077c78c"}
Jul 03 09:54:12.314807 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.314489    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7qhwb"
Jul 03 09:54:12.332002 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.331957    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7qhwb" podStartSLOduration=2.440010376 podStartE2EDuration="4.331941778s" totalImagesPullingTime="1.891931402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:54:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:54:12.331336911 +0000 UTC m=+654.196625566" watchObservedRunningTime="2026-07-03 09:54:12.331941778 +0000 UTC m=+654.197230433"
Jul 03 09:54:12.562387 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.562351    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"]
Jul 03 09:54:12.566574 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.566525    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"
Jul 03 09:54:12.569313 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.569288    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 03 09:54:12.569448 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.569292    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 03 09:54:12.569448 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.569292    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-phr6p\""
Jul 03 09:54:12.575434 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.575412    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"]
Jul 03 09:54:12.578062 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.578021    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:12.654538 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.654508    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t24ls\" (UniqueName: \"kubernetes.io/projected/3b350445-4582-4d07-a021-256d60bb143e-kube-api-access-t24ls\") pod \"kuadrant-console-plugin-66b5774985-x8ktz\" (UID: \"3b350445-4582-4d07-a021-256d60bb143e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"
Jul 03 09:54:12.654696 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.654576    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b350445-4582-4d07-a021-256d60bb143e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-x8ktz\" (UID: \"3b350445-4582-4d07-a021-256d60bb143e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"
Jul 03 09:54:12.654696 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.654680    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b350445-4582-4d07-a021-256d60bb143e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x8ktz\" (UID: \"3b350445-4582-4d07-a021-256d60bb143e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"
Jul 03 09:54:12.755846 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.755806    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t24ls\" (UniqueName: \"kubernetes.io/projected/3b350445-4582-4d07-a021-256d60bb143e-kube-api-access-t24ls\") pod \"kuadrant-console-plugin-66b5774985-x8ktz\" (UID: \"3b350445-4582-4d07-a021-256d60bb143e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"
Jul 03 09:54:12.756006 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.755925    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b350445-4582-4d07-a021-256d60bb143e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-x8ktz\" (UID: \"3b350445-4582-4d07-a021-256d60bb143e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"
Jul 03 09:54:12.756006 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.755975    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b350445-4582-4d07-a021-256d60bb143e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x8ktz\" (UID: \"3b350445-4582-4d07-a021-256d60bb143e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"
Jul 03 09:54:12.756127 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:12.756111    2571 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 03 09:54:12.756193 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:12.756180    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b350445-4582-4d07-a021-256d60bb143e-plugin-serving-cert podName:3b350445-4582-4d07-a021-256d60bb143e nodeName:}" failed. No retries permitted until 2026-07-03 09:54:13.256160284 +0000 UTC m=+655.121448931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/3b350445-4582-4d07-a021-256d60bb143e-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-x8ktz" (UID: "3b350445-4582-4d07-a021-256d60bb143e") : secret "plugin-serving-cert" not found
Jul 03 09:54:12.756661 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.756641    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b350445-4582-4d07-a021-256d60bb143e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-x8ktz\" (UID: \"3b350445-4582-4d07-a021-256d60bb143e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"
Jul 03 09:54:12.768926 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.768894    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t24ls\" (UniqueName: \"kubernetes.io/projected/3b350445-4582-4d07-a021-256d60bb143e-kube-api-access-t24ls\") pod \"kuadrant-console-plugin-66b5774985-x8ktz\" (UID: \"3b350445-4582-4d07-a021-256d60bb143e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"
Jul 03 09:54:12.885061 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:12.884963    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:13.227290 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:13.227252    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:13.261078 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:13.261048    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b350445-4582-4d07-a021-256d60bb143e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x8ktz\" (UID: \"3b350445-4582-4d07-a021-256d60bb143e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"
Jul 03 09:54:13.263543 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:13.263513    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b350445-4582-4d07-a021-256d60bb143e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x8ktz\" (UID: \"3b350445-4582-4d07-a021-256d60bb143e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"
Jul 03 09:54:13.284753 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:13.284725    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:13.477662 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:13.477582    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"
Jul 03 09:54:13.615395 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:13.615370    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz"]
Jul 03 09:54:13.617726 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:54:13.617700    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b350445_4582_4d07_a021_256d60bb143e.slice/crio-b29077c9ead5b61cf41ee2a7ad0214066ee74a4c68a9e2d4a0f3a8aea87904cd WatchSource:0}: Error finding container b29077c9ead5b61cf41ee2a7ad0214066ee74a4c68a9e2d4a0f3a8aea87904cd: Status 404 returned error can't find the container with id b29077c9ead5b61cf41ee2a7ad0214066ee74a4c68a9e2d4a0f3a8aea87904cd
Jul 03 09:54:14.257982 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:14.257952    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6wng7"
Jul 03 09:54:14.326574 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:14.326533    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz" event={"ID":"3b350445-4582-4d07-a021-256d60bb143e","Type":"ContainerStarted","Data":"b29077c9ead5b61cf41ee2a7ad0214066ee74a4c68a9e2d4a0f3a8aea87904cd"}
Jul 03 09:54:14.579550 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:14.579479    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:14.664995 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:14.664957    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:20.833161 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:20.833122    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:20.937452 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:20.937412    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:21.306948 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:21.306921    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:21.307145 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:21.306993    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:23.151475 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.151435    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"]
Jul 03 09:54:23.151991 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.151702    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj" podUID="640c6872-5609-4aa6-b3bf-213fda077942" containerName="manager" containerID="cri-o://f6c12d915f1b3156e0d2cba83d19794fda49be0b0f137ce95cbccf6900b5ced7" gracePeriod=2
Jul 03 09:54:23.161199 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.161124    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"]
Jul 03 09:54:23.175413 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.175375    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"]
Jul 03 09:54:23.175662 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.175611    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7" podUID="bb21412f-51de-4857-a799-0fcbf11f585e" containerName="manager" containerID="cri-o://0e4fce182e923e02e282c584b1009dba920317f82cab30e21050db9e7be96e00" gracePeriod=2
Jul 03 09:54:23.179590 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:23.179550    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="640c6872-5609-4aa6-b3bf-213fda077942" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:23.190808 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.190774    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"]
Jul 03 09:54:23.191407 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.191367    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="640c6872-5609-4aa6-b3bf-213fda077942" containerName="manager"
Jul 03 09:54:23.191407 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.191393    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="640c6872-5609-4aa6-b3bf-213fda077942" containerName="manager"
Jul 03 09:54:23.191579 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.191487    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="640c6872-5609-4aa6-b3bf-213fda077942" containerName="manager"
Jul 03 09:54:23.195761 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.195736    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"
Jul 03 09:54:23.197762 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.197709    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"]
Jul 03 09:54:23.207866 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:23.207643    2571 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bb21412f-51de-4857-a799-0fcbf11f585e\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-zb6x7\": pods \"limitador-operator-controller-manager-859d7b78bd-zb6x7\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:23.209988 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:23.209958    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="640c6872-5609-4aa6-b3bf-213fda077942" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:23.212880 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:23.212529    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zb6x7\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="bb21412f-51de-4857-a799-0fcbf11f585e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:23.216404 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.216373    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"]
Jul 03 09:54:23.218696 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.218657    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x4qtq"]
Jul 03 09:54:23.219142 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.219118    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb21412f-51de-4857-a799-0fcbf11f585e" containerName="manager"
Jul 03 09:54:23.219142 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.219141    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb21412f-51de-4857-a799-0fcbf11f585e" containerName="manager"
Jul 03 09:54:23.219306 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.219220    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb21412f-51de-4857-a799-0fcbf11f585e" containerName="manager"
Jul 03 09:54:23.222863 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.222755    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x4qtq"
Jul 03 09:54:23.239236 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.239201    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x4qtq"]
Jul 03 09:54:23.244951 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:23.244905    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="640c6872-5609-4aa6-b3bf-213fda077942" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:23.247632 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:23.247590    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zb6x7\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="bb21412f-51de-4857-a799-0fcbf11f585e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:23.254480 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.254447    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15c625df-b4fc-4ef9-9f86-6d587a1cb0a4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-frdsb\" (UID: \"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"
Jul 03 09:54:23.254618 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.254603    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m49mc\" (UniqueName: \"kubernetes.io/projected/7848c387-6f31-4a59-b5cd-be0a154dbf04-kube-api-access-m49mc\") pod \"limitador-operator-controller-manager-859d7b78bd-x4qtq\" (UID: \"7848c387-6f31-4a59-b5cd-be0a154dbf04\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x4qtq"
Jul 03 09:54:23.254691 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.254637    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk5xk\" (UniqueName: \"kubernetes.io/projected/15c625df-b4fc-4ef9-9f86-6d587a1cb0a4-kube-api-access-vk5xk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-frdsb\" (UID: \"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"
Jul 03 09:54:23.321898 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.321867    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7qhwb"
Jul 03 09:54:23.324201 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:23.324161    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="640c6872-5609-4aa6-b3bf-213fda077942" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:23.341623 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.341587    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:23.354614 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:23.354567    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zb6x7\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="bb21412f-51de-4857-a799-0fcbf11f585e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:23.356011 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.355983    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15c625df-b4fc-4ef9-9f86-6d587a1cb0a4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-frdsb\" (UID: \"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"
Jul 03 09:54:23.356135 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.356088    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m49mc\" (UniqueName: \"kubernetes.io/projected/7848c387-6f31-4a59-b5cd-be0a154dbf04-kube-api-access-m49mc\") pod \"limitador-operator-controller-manager-859d7b78bd-x4qtq\" (UID: \"7848c387-6f31-4a59-b5cd-be0a154dbf04\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x4qtq"
Jul 03 09:54:23.356135 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.356119    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk5xk\" (UniqueName: \"kubernetes.io/projected/15c625df-b4fc-4ef9-9f86-6d587a1cb0a4-kube-api-access-vk5xk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-frdsb\" (UID: \"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"
Jul 03 09:54:23.356483 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.356454    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15c625df-b4fc-4ef9-9f86-6d587a1cb0a4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-frdsb\" (UID: \"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"
Jul 03 09:54:23.370177 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.370143    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m49mc\" (UniqueName: \"kubernetes.io/projected/7848c387-6f31-4a59-b5cd-be0a154dbf04-kube-api-access-m49mc\") pod \"limitador-operator-controller-manager-859d7b78bd-x4qtq\" (UID: \"7848c387-6f31-4a59-b5cd-be0a154dbf04\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x4qtq"
Jul 03 09:54:23.372020 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.371988    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk5xk\" (UniqueName: \"kubernetes.io/projected/15c625df-b4fc-4ef9-9f86-6d587a1cb0a4-kube-api-access-vk5xk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-frdsb\" (UID: \"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"
Jul 03 09:54:23.447253 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.447166    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:23.576972 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.576931    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"
Jul 03 09:54:23.584039 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:23.584007    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x4qtq"
Jul 03 09:54:24.418913 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:24.418874    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"]
Jul 03 09:54:24.433561 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:24.433529    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"
Jul 03 09:54:24.436312 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:24.436275    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zb6x7\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="bb21412f-51de-4857-a799-0fcbf11f585e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:24.438538 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:24.438501    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="640c6872-5609-4aa6-b3bf-213fda077942" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:24.439964 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:24.439939    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"]
Jul 03 09:54:24.464265 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:24.464225    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e62bbfb4-22a3-47eb-b82b-4a646b93ce57-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bvwq2\" (UID: \"e62bbfb4-22a3-47eb-b82b-4a646b93ce57\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"
Jul 03 09:54:24.464432 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:24.464307    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwppr\" (UniqueName: \"kubernetes.io/projected/e62bbfb4-22a3-47eb-b82b-4a646b93ce57-kube-api-access-bwppr\") pod \"kuadrant-operator-controller-manager-86878c769f-bvwq2\" (UID: \"e62bbfb4-22a3-47eb-b82b-4a646b93ce57\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"
Jul 03 09:54:24.565503 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:24.565458    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e62bbfb4-22a3-47eb-b82b-4a646b93ce57-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bvwq2\" (UID: \"e62bbfb4-22a3-47eb-b82b-4a646b93ce57\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"
Jul 03 09:54:24.565678 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:24.565527    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwppr\" (UniqueName: \"kubernetes.io/projected/e62bbfb4-22a3-47eb-b82b-4a646b93ce57-kube-api-access-bwppr\") pod \"kuadrant-operator-controller-manager-86878c769f-bvwq2\" (UID: \"e62bbfb4-22a3-47eb-b82b-4a646b93ce57\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"
Jul 03 09:54:24.565949 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:24.565921    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e62bbfb4-22a3-47eb-b82b-4a646b93ce57-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bvwq2\" (UID: \"e62bbfb4-22a3-47eb-b82b-4a646b93ce57\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"
Jul 03 09:54:24.579347 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:24.579310    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwppr\" (UniqueName: \"kubernetes.io/projected/e62bbfb4-22a3-47eb-b82b-4a646b93ce57-kube-api-access-bwppr\") pod \"kuadrant-operator-controller-manager-86878c769f-bvwq2\" (UID: \"e62bbfb4-22a3-47eb-b82b-4a646b93ce57\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"
Jul 03 09:54:24.747612 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:24.747520    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"
Jul 03 09:54:25.386693 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:25.386646    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-764c6c8597-qcgk8" podUID="7ce6f509-c01e-40df-b63b-51bced4bdb52" containerName="console" containerID="cri-o://65ff00ef2365097c934fa9f8eab3a9f1aaa83e1f34cc2101f82b572370ef4afb" gracePeriod=15
Jul 03 09:54:28.838014 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:28.837976    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:28.942202 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:28.942138    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:29.483305 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:29.483271    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:29.585737 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:29.585707    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:33.261232 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:33.261183    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:33.330804 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:33.330768    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:34.087915 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:34.087885    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:34.178762 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:34.178404    2571 patch_prober.go:28] interesting pod/console-764c6c8597-qcgk8 container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.133.0.22:8443/health\": dial tcp 10.133.0.22:8443: connect: connection refused" start-of-body=
Jul 03 09:54:34.178762 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:34.178463    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-764c6c8597-qcgk8" podUID="7ce6f509-c01e-40df-b63b-51bced4bdb52" containerName="console" probeResult="failure" output="Get \"https://10.133.0.22:8443/health\": dial tcp 10.133.0.22:8443: connect: connection refused"
Jul 03 09:54:36.472325 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.472188    2571 generic.go:356] "Generic (PLEG): container finished" podID="bb21412f-51de-4857-a799-0fcbf11f585e" containerID="0e4fce182e923e02e282c584b1009dba920317f82cab30e21050db9e7be96e00" exitCode=0
Jul 03 09:54:36.475171 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.475138    2571 generic.go:356] "Generic (PLEG): container finished" podID="640c6872-5609-4aa6-b3bf-213fda077942" containerID="f6c12d915f1b3156e0d2cba83d19794fda49be0b0f137ce95cbccf6900b5ced7" exitCode=0
Jul 03 09:54:36.478011 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.477802    2571 generic.go:356] "Generic (PLEG): container finished" podID="7ce6f509-c01e-40df-b63b-51bced4bdb52" containerID="65ff00ef2365097c934fa9f8eab3a9f1aaa83e1f34cc2101f82b572370ef4afb" exitCode=0
Jul 03 09:54:36.478011 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.477941    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-764c6c8597-qcgk8" event={"ID":"7ce6f509-c01e-40df-b63b-51bced4bdb52","Type":"ContainerDied","Data":"65ff00ef2365097c934fa9f8eab3a9f1aaa83e1f34cc2101f82b572370ef4afb"}
Jul 03 09:54:36.620479 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.620447    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:54:36.622634 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.622615    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:36.629765 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.629737    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"]
Jul 03 09:54:36.631845 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:54:36.631801    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode62bbfb4_22a3_47eb_b82b_4a646b93ce57.slice/crio-2fbf5e1ab53cddbb9bb0ee10950c62b4bfa0e6ead33ffc55fea34d17339b8e7d WatchSource:0}: Error finding container 2fbf5e1ab53cddbb9bb0ee10950c62b4bfa0e6ead33ffc55fea34d17339b8e7d: Status 404 returned error can't find the container with id 2fbf5e1ab53cddbb9bb0ee10950c62b4bfa0e6ead33ffc55fea34d17339b8e7d
Jul 03 09:54:36.633711 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.633664    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:36.645196 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.645122    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x4qtq"]
Jul 03 09:54:36.649294 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:36.649225    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="640c6872-5609-4aa6-b3bf-213fda077942" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:36.651346 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:54:36.651302    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7848c387_6f31_4a59_b5cd_be0a154dbf04.slice/crio-4e0e2423d5ae2304433cba241c4cf7d02dde91a4a7bde13137e484e54bc7cf0d WatchSource:0}: Error finding container 4e0e2423d5ae2304433cba241c4cf7d02dde91a4a7bde13137e484e54bc7cf0d: Status 404 returned error can't find the container with id 4e0e2423d5ae2304433cba241c4cf7d02dde91a4a7bde13137e484e54bc7cf0d
Jul 03 09:54:36.651855 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:36.651785    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zb6x7\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="bb21412f-51de-4857-a799-0fcbf11f585e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:36.677366 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.677336    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"]
Jul 03 09:54:36.682408 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.682387    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-config\") pod \"7ce6f509-c01e-40df-b63b-51bced4bdb52\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") "
Jul 03 09:54:36.682499 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.682436    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/640c6872-5609-4aa6-b3bf-213fda077942-extensions-socket-volume\") pod \"640c6872-5609-4aa6-b3bf-213fda077942\" (UID: \"640c6872-5609-4aa6-b3bf-213fda077942\") "
Jul 03 09:54:36.682499 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.682458    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwsz7\" (UniqueName: \"kubernetes.io/projected/bb21412f-51de-4857-a799-0fcbf11f585e-kube-api-access-gwsz7\") pod \"bb21412f-51de-4857-a799-0fcbf11f585e\" (UID: \"bb21412f-51de-4857-a799-0fcbf11f585e\") "
Jul 03 09:54:36.682499 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.682480    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkzl2\" (UniqueName: \"kubernetes.io/projected/640c6872-5609-4aa6-b3bf-213fda077942-kube-api-access-qkzl2\") pod \"640c6872-5609-4aa6-b3bf-213fda077942\" (UID: \"640c6872-5609-4aa6-b3bf-213fda077942\") "
Jul 03 09:54:36.682661 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.682506    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-oauth-serving-cert\") pod \"7ce6f509-c01e-40df-b63b-51bced4bdb52\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") "
Jul 03 09:54:36.682661 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.682526    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-trusted-ca-bundle\") pod \"7ce6f509-c01e-40df-b63b-51bced4bdb52\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") "
Jul 03 09:54:36.682661 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.682548    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-service-ca\") pod \"7ce6f509-c01e-40df-b63b-51bced4bdb52\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") "
Jul 03 09:54:36.682661 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.682618    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-oauth-config\") pod \"7ce6f509-c01e-40df-b63b-51bced4bdb52\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") "
Jul 03 09:54:36.682886 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.682675    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-serving-cert\") pod \"7ce6f509-c01e-40df-b63b-51bced4bdb52\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") "
Jul 03 09:54:36.682886 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.682713    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw45r\" (UniqueName: \"kubernetes.io/projected/7ce6f509-c01e-40df-b63b-51bced4bdb52-kube-api-access-mw45r\") pod \"7ce6f509-c01e-40df-b63b-51bced4bdb52\" (UID: \"7ce6f509-c01e-40df-b63b-51bced4bdb52\") "
Jul 03 09:54:36.682886 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.682725    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-config" (OuterVolumeSpecName: "console-config") pod "7ce6f509-c01e-40df-b63b-51bced4bdb52" (UID: "7ce6f509-c01e-40df-b63b-51bced4bdb52"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 09:54:36.683042 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.682961    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7ce6f509-c01e-40df-b63b-51bced4bdb52" (UID: "7ce6f509-c01e-40df-b63b-51bced4bdb52"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 09:54:36.683042 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.682968    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/640c6872-5609-4aa6-b3bf-213fda077942-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "640c6872-5609-4aa6-b3bf-213fda077942" (UID: "640c6872-5609-4aa6-b3bf-213fda077942"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 09:54:36.683144 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.683047    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-config\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:54:36.683144 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.683065    2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/640c6872-5609-4aa6-b3bf-213fda077942-extensions-socket-volume\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:54:36.683144 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.683079    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-oauth-serving-cert\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:54:36.683144 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.683097    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-service-ca" (OuterVolumeSpecName: "service-ca") pod "7ce6f509-c01e-40df-b63b-51bced4bdb52" (UID: "7ce6f509-c01e-40df-b63b-51bced4bdb52"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 09:54:36.683343 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.683317    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7ce6f509-c01e-40df-b63b-51bced4bdb52" (UID: "7ce6f509-c01e-40df-b63b-51bced4bdb52"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 09:54:36.684975 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.684935    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7ce6f509-c01e-40df-b63b-51bced4bdb52" (UID: "7ce6f509-c01e-40df-b63b-51bced4bdb52"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 09:54:36.685071 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.685005    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/640c6872-5609-4aa6-b3bf-213fda077942-kube-api-access-qkzl2" (OuterVolumeSpecName: "kube-api-access-qkzl2") pod "640c6872-5609-4aa6-b3bf-213fda077942" (UID: "640c6872-5609-4aa6-b3bf-213fda077942"). InnerVolumeSpecName "kube-api-access-qkzl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:54:36.685071 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.685048    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb21412f-51de-4857-a799-0fcbf11f585e-kube-api-access-gwsz7" (OuterVolumeSpecName: "kube-api-access-gwsz7") pod "bb21412f-51de-4857-a799-0fcbf11f585e" (UID: "bb21412f-51de-4857-a799-0fcbf11f585e"). InnerVolumeSpecName "kube-api-access-gwsz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:54:36.685311 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.685286    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7ce6f509-c01e-40df-b63b-51bced4bdb52" (UID: "7ce6f509-c01e-40df-b63b-51bced4bdb52"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 09:54:36.685370 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.685310    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce6f509-c01e-40df-b63b-51bced4bdb52-kube-api-access-mw45r" (OuterVolumeSpecName: "kube-api-access-mw45r") pod "7ce6f509-c01e-40df-b63b-51bced4bdb52" (UID: "7ce6f509-c01e-40df-b63b-51bced4bdb52"). InnerVolumeSpecName "kube-api-access-mw45r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:54:36.692892 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:54:36.692852    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c625df_b4fc_4ef9_9f86_6d587a1cb0a4.slice/crio-981a19fd12ab3e2020060080aa9e2fdc908cdbec1e9c41dfd52888894e69bb80 WatchSource:0}: Error finding container 981a19fd12ab3e2020060080aa9e2fdc908cdbec1e9c41dfd52888894e69bb80: Status 404 returned error can't find the container with id 981a19fd12ab3e2020060080aa9e2fdc908cdbec1e9c41dfd52888894e69bb80
Jul 03 09:54:36.765620 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.765596    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="640c6872-5609-4aa6-b3bf-213fda077942" path="/var/lib/kubelet/pods/640c6872-5609-4aa6-b3bf-213fda077942/volumes"
Jul 03 09:54:36.765955 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.765942    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb21412f-51de-4857-a799-0fcbf11f585e" path="/var/lib/kubelet/pods/bb21412f-51de-4857-a799-0fcbf11f585e/volumes"
Jul 03 09:54:36.783957 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.783816    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw45r\" (UniqueName: \"kubernetes.io/projected/7ce6f509-c01e-40df-b63b-51bced4bdb52-kube-api-access-mw45r\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:54:36.783957 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.783868    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwsz7\" (UniqueName: \"kubernetes.io/projected/bb21412f-51de-4857-a799-0fcbf11f585e-kube-api-access-gwsz7\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:54:36.783957 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.783878    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkzl2\" (UniqueName: \"kubernetes.io/projected/640c6872-5609-4aa6-b3bf-213fda077942-kube-api-access-qkzl2\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:54:36.783957 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.783887    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-trusted-ca-bundle\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:54:36.783957 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.783895    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce6f509-c01e-40df-b63b-51bced4bdb52-service-ca\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:54:36.783957 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.783903    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-oauth-config\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:54:36.783957 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:36.783912    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ce6f509-c01e-40df-b63b-51bced4bdb52-console-serving-cert\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:54:37.485456 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.485419    2571 scope.go:117] "RemoveContainer" containerID="f6c12d915f1b3156e0d2cba83d19794fda49be0b0f137ce95cbccf6900b5ced7"
Jul 03 09:54:37.485963 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.485588    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:37.488585 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:37.488552    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="640c6872-5609-4aa6-b3bf-213fda077942" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:37.490393 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.490353    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x4qtq" event={"ID":"7848c387-6f31-4a59-b5cd-be0a154dbf04","Type":"ContainerStarted","Data":"33465171ff279ade92bdfc507f2bbf24e6c4a2ce0ed69ae6025e2813ea37c70c"}
Jul 03 09:54:37.490504 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.490454    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x4qtq"
Jul 03 09:54:37.490504 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.490482    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x4qtq" event={"ID":"7848c387-6f31-4a59-b5cd-be0a154dbf04","Type":"ContainerStarted","Data":"4e0e2423d5ae2304433cba241c4cf7d02dde91a4a7bde13137e484e54bc7cf0d"}
Jul 03 09:54:37.491989 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:37.491963    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="640c6872-5609-4aa6-b3bf-213fda077942" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:37.492155 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.492036    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-764c6c8597-qcgk8"
Jul 03 09:54:37.492155 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.492064    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-764c6c8597-qcgk8" event={"ID":"7ce6f509-c01e-40df-b63b-51bced4bdb52","Type":"ContainerDied","Data":"5845bf67e348d49cf48401a3a44f8427c3a6e1c6a2d1a8b8cff9ef486453cbf2"}
Jul 03 09:54:37.493727 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.493664    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb" event={"ID":"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4","Type":"ContainerStarted","Data":"5c3ba0b03f66d6209cfb16af270c50e830db711d00f138c5b8ad53aeaa262ae5"}
Jul 03 09:54:37.493727 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.493717    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb" event={"ID":"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4","Type":"ContainerStarted","Data":"981a19fd12ab3e2020060080aa9e2fdc908cdbec1e9c41dfd52888894e69bb80"}
Jul 03 09:54:37.493917 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.493746    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"
Jul 03 09:54:37.495608 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.495585    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2" event={"ID":"e62bbfb4-22a3-47eb-b82b-4a646b93ce57","Type":"ContainerStarted","Data":"58efb3e8debdbe5e4ad280d7b62b77c26bb9b5006819f8ee0b37c8604ca28d62"}
Jul 03 09:54:37.495708 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.495628    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2" event={"ID":"e62bbfb4-22a3-47eb-b82b-4a646b93ce57","Type":"ContainerStarted","Data":"2fbf5e1ab53cddbb9bb0ee10950c62b4bfa0e6ead33ffc55fea34d17339b8e7d"}
Jul 03 09:54:37.495708 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.495655    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"
Jul 03 09:54:37.496892 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.496877    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:37.498414 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.498389    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz" event={"ID":"3b350445-4582-4d07-a021-256d60bb143e","Type":"ContainerStarted","Data":"5659150dbb79b16a951e569cfd514e7dbcb41a9c1b9a1aad6e091e64d8b756be"}
Jul 03 09:54:37.499963 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.499947    2571 scope.go:117] "RemoveContainer" containerID="65ff00ef2365097c934fa9f8eab3a9f1aaa83e1f34cc2101f82b572370ef4afb"
Jul 03 09:54:37.508506 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.508468    2571 scope.go:117] "RemoveContainer" containerID="0e4fce182e923e02e282c584b1009dba920317f82cab30e21050db9e7be96e00"
Jul 03 09:54:37.509936 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.509898    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x4qtq" podStartSLOduration=14.509884646 podStartE2EDuration="14.509884646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:54:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:54:37.508604165 +0000 UTC m=+679.373892821" watchObservedRunningTime="2026-07-03 09:54:37.509884646 +0000 UTC m=+679.375173300"
Jul 03 09:54:37.510629 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:37.510607    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="640c6872-5609-4aa6-b3bf-213fda077942" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:37.512606 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:37.512582    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="640c6872-5609-4aa6-b3bf-213fda077942" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:37.529963 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.529912    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x8ktz" podStartSLOduration=2.635874793 podStartE2EDuration="25.529894542s" totalImagesPullingTime="22.894019749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:54:37.528426174 +0000 UTC m=+679.393714831" watchObservedRunningTime="2026-07-03 09:54:37.529894542 +0000 UTC m=+679.395183196"
Jul 03 09:54:37.548014 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.547953    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2" podStartSLOduration=13.547931064 podStartE2EDuration="13.547931064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:54:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:54:37.546447747 +0000 UTC m=+679.411736430" watchObservedRunningTime="2026-07-03 09:54:37.547931064 +0000 UTC m=+679.413219720"
Jul 03 09:54:37.561731 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.561698    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-764c6c8597-qcgk8"]
Jul 03 09:54:37.565250 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:37.565198    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zb6x7\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="bb21412f-51de-4857-a799-0fcbf11f585e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:37.565410 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.565389    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-764c6c8597-qcgk8"]
Jul 03 09:54:37.587206 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:37.587155    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb" podStartSLOduration=14.587139161 podStartE2EDuration="14.587139161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:54:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:54:37.585050488 +0000 UTC m=+679.450339146" watchObservedRunningTime="2026-07-03 09:54:37.587139161 +0000 UTC m=+679.452427816"
Jul 03 09:54:38.763261 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:38.763222    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vbhtj\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="640c6872-5609-4aa6-b3bf-213fda077942" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vbhtj"
Jul 03 09:54:38.765282 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:38.765254    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-zb6x7\" is forbidden: User \"system:node:ip-10-0-138-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-129.ec2.internal' and this object" podUID="bb21412f-51de-4857-a799-0fcbf11f585e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zb6x7"
Jul 03 09:54:38.765649 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:38.765631    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ce6f509-c01e-40df-b63b-51bced4bdb52" path="/var/lib/kubelet/pods/7ce6f509-c01e-40df-b63b-51bced4bdb52/volumes"
Jul 03 09:54:39.845524 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:39.845490    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:39.946783 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:39.946751    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:42.305126 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:42.305085    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:42.407426 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:42.407391    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:54:48.509647 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:48.509616    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"
Jul 03 09:54:48.510052 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:48.509701    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x4qtq"
Jul 03 09:54:48.510052 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:48.509748    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"
Jul 03 09:54:48.571531 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:48.571497    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"]
Jul 03 09:54:48.571726 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:48.571706    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb" podUID="15c625df-b4fc-4ef9-9f86-6d587a1cb0a4" containerName="manager" containerID="cri-o://5c3ba0b03f66d6209cfb16af270c50e830db711d00f138c5b8ad53aeaa262ae5" gracePeriod=10
Jul 03 09:54:48.817971 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:48.817936    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"
Jul 03 09:54:48.891854 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:48.891807    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vk5xk\" (UniqueName: \"kubernetes.io/projected/15c625df-b4fc-4ef9-9f86-6d587a1cb0a4-kube-api-access-vk5xk\") pod \"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4\" (UID: \"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4\") "
Jul 03 09:54:48.892049 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:48.891891    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15c625df-b4fc-4ef9-9f86-6d587a1cb0a4-extensions-socket-volume\") pod \"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4\" (UID: \"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4\") "
Jul 03 09:54:48.892310 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:48.892286    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15c625df-b4fc-4ef9-9f86-6d587a1cb0a4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "15c625df-b4fc-4ef9-9f86-6d587a1cb0a4" (UID: "15c625df-b4fc-4ef9-9f86-6d587a1cb0a4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 09:54:48.893932 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:48.893911    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15c625df-b4fc-4ef9-9f86-6d587a1cb0a4-kube-api-access-vk5xk" (OuterVolumeSpecName: "kube-api-access-vk5xk") pod "15c625df-b4fc-4ef9-9f86-6d587a1cb0a4" (UID: "15c625df-b4fc-4ef9-9f86-6d587a1cb0a4"). InnerVolumeSpecName "kube-api-access-vk5xk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:54:48.993190 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:48.993163    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk5xk\" (UniqueName: \"kubernetes.io/projected/15c625df-b4fc-4ef9-9f86-6d587a1cb0a4-kube-api-access-vk5xk\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:54:48.993190 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:48.993191    2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15c625df-b4fc-4ef9-9f86-6d587a1cb0a4-extensions-socket-volume\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:54:49.565456 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:49.565420    2571 generic.go:356] "Generic (PLEG): container finished" podID="15c625df-b4fc-4ef9-9f86-6d587a1cb0a4" containerID="5c3ba0b03f66d6209cfb16af270c50e830db711d00f138c5b8ad53aeaa262ae5" exitCode=0
Jul 03 09:54:49.565880 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:49.565463    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"
Jul 03 09:54:49.565880 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:49.565493    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb" event={"ID":"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4","Type":"ContainerDied","Data":"5c3ba0b03f66d6209cfb16af270c50e830db711d00f138c5b8ad53aeaa262ae5"}
Jul 03 09:54:49.565880 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:49.565544    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb" event={"ID":"15c625df-b4fc-4ef9-9f86-6d587a1cb0a4","Type":"ContainerDied","Data":"981a19fd12ab3e2020060080aa9e2fdc908cdbec1e9c41dfd52888894e69bb80"}
Jul 03 09:54:49.565880 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:49.565571    2571 scope.go:117] "RemoveContainer" containerID="5c3ba0b03f66d6209cfb16af270c50e830db711d00f138c5b8ad53aeaa262ae5"
Jul 03 09:54:49.574728 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:49.574549    2571 scope.go:117] "RemoveContainer" containerID="5c3ba0b03f66d6209cfb16af270c50e830db711d00f138c5b8ad53aeaa262ae5"
Jul 03 09:54:49.574885 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:54:49.574867    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c3ba0b03f66d6209cfb16af270c50e830db711d00f138c5b8ad53aeaa262ae5\": container with ID starting with 5c3ba0b03f66d6209cfb16af270c50e830db711d00f138c5b8ad53aeaa262ae5 not found: ID does not exist" containerID="5c3ba0b03f66d6209cfb16af270c50e830db711d00f138c5b8ad53aeaa262ae5"
Jul 03 09:54:49.574938 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:49.574895    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3ba0b03f66d6209cfb16af270c50e830db711d00f138c5b8ad53aeaa262ae5"} err="failed to get container status \"5c3ba0b03f66d6209cfb16af270c50e830db711d00f138c5b8ad53aeaa262ae5\": rpc error: code = NotFound desc = could not find container \"5c3ba0b03f66d6209cfb16af270c50e830db711d00f138c5b8ad53aeaa262ae5\": container with ID starting with 5c3ba0b03f66d6209cfb16af270c50e830db711d00f138c5b8ad53aeaa262ae5 not found: ID does not exist"
Jul 03 09:54:49.591861 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:49.591820    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"]
Jul 03 09:54:49.598011 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:49.597985    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-frdsb"]
Jul 03 09:54:50.764372 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:54:50.764329    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15c625df-b4fc-4ef9-9f86-6d587a1cb0a4" path="/var/lib/kubelet/pods/15c625df-b4fc-4ef9-9f86-6d587a1cb0a4/volumes"
Jul 03 09:55:01.111408 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:01.111371    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:04.963636 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:04.963602    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:05.174348 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:05.174316    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:10.339102 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.339023    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:10.448927 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.448890    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:10.531059 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.531025    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:10.627451 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.627356    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:55:10.627889 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.627872    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce6f509-c01e-40df-b63b-51bced4bdb52" containerName="console"
Jul 03 09:55:10.627997 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.627891    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce6f509-c01e-40df-b63b-51bced4bdb52" containerName="console"
Jul 03 09:55:10.627997 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.627918    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15c625df-b4fc-4ef9-9f86-6d587a1cb0a4" containerName="manager"
Jul 03 09:55:10.627997 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.627926    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="15c625df-b4fc-4ef9-9f86-6d587a1cb0a4" containerName="manager"
Jul 03 09:55:10.628146 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.628037    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce6f509-c01e-40df-b63b-51bced4bdb52" containerName="console"
Jul 03 09:55:10.628146 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.628052    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="15c625df-b4fc-4ef9-9f86-6d587a1cb0a4" containerName="manager"
Jul 03 09:55:10.673773 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.673737    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:55:10.673994 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.673784    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:55:10.673994 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.673910    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"
Jul 03 09:55:10.677071 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.677043    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 03 09:55:10.680770 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.680743    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/84b5540a-d878-43fd-9154-3d431171bf7c-config-file\") pod \"limitador-limitador-54d6d988d5-2lnjn\" (UID: \"84b5540a-d878-43fd-9154-3d431171bf7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"
Jul 03 09:55:10.680911 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.680787    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtr5f\" (UniqueName: \"kubernetes.io/projected/84b5540a-d878-43fd-9154-3d431171bf7c-kube-api-access-wtr5f\") pod \"limitador-limitador-54d6d988d5-2lnjn\" (UID: \"84b5540a-d878-43fd-9154-3d431171bf7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"
Jul 03 09:55:10.782227 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.782188    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/84b5540a-d878-43fd-9154-3d431171bf7c-config-file\") pod \"limitador-limitador-54d6d988d5-2lnjn\" (UID: \"84b5540a-d878-43fd-9154-3d431171bf7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"
Jul 03 09:55:10.782403 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.782244    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtr5f\" (UniqueName: \"kubernetes.io/projected/84b5540a-d878-43fd-9154-3d431171bf7c-kube-api-access-wtr5f\") pod \"limitador-limitador-54d6d988d5-2lnjn\" (UID: \"84b5540a-d878-43fd-9154-3d431171bf7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"
Jul 03 09:55:10.782888 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.782869    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/84b5540a-d878-43fd-9154-3d431171bf7c-config-file\") pod \"limitador-limitador-54d6d988d5-2lnjn\" (UID: \"84b5540a-d878-43fd-9154-3d431171bf7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"
Jul 03 09:55:10.790279 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.790248    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtr5f\" (UniqueName: \"kubernetes.io/projected/84b5540a-d878-43fd-9154-3d431171bf7c-kube-api-access-wtr5f\") pod \"limitador-limitador-54d6d988d5-2lnjn\" (UID: \"84b5540a-d878-43fd-9154-3d431171bf7c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"
Jul 03 09:55:10.985603 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:10.985574    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"
Jul 03 09:55:11.125503 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.125475    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:55:11.128156 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:55:11.128129    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84b5540a_d878_43fd_9154_3d431171bf7c.slice/crio-3d249f763fe84fed5dbb3ef492a069bd9012a6ff0a562d7f0f427570eed1c4bc WatchSource:0}: Error finding container 3d249f763fe84fed5dbb3ef492a069bd9012a6ff0a562d7f0f427570eed1c4bc: Status 404 returned error can't find the container with id 3d249f763fe84fed5dbb3ef492a069bd9012a6ff0a562d7f0f427570eed1c4bc
Jul 03 09:55:11.172261 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.172220    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-zh8lr"]
Jul 03 09:55:11.175356 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.175331    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zh8lr"
Jul 03 09:55:11.178528 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.178502    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rngsf\""
Jul 03 09:55:11.184804 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.184757    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-zh8lr"]
Jul 03 09:55:11.185024 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.185006    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbk2q\" (UniqueName: \"kubernetes.io/projected/9f06c156-5631-446c-a4d0-ab01adf84c5b-kube-api-access-zbk2q\") pod \"authorino-58cd955675-zh8lr\" (UID: \"9f06c156-5631-446c-a4d0-ab01adf84c5b\") " pod="kuadrant-system/authorino-58cd955675-zh8lr"
Jul 03 09:55:11.212063 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.212028    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:11.285470 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.285391    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbk2q\" (UniqueName: \"kubernetes.io/projected/9f06c156-5631-446c-a4d0-ab01adf84c5b-kube-api-access-zbk2q\") pod \"authorino-58cd955675-zh8lr\" (UID: \"9f06c156-5631-446c-a4d0-ab01adf84c5b\") " pod="kuadrant-system/authorino-58cd955675-zh8lr"
Jul 03 09:55:11.295123 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.295093    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbk2q\" (UniqueName: \"kubernetes.io/projected/9f06c156-5631-446c-a4d0-ab01adf84c5b-kube-api-access-zbk2q\") pod \"authorino-58cd955675-zh8lr\" (UID: \"9f06c156-5631-446c-a4d0-ab01adf84c5b\") " pod="kuadrant-system/authorino-58cd955675-zh8lr"
Jul 03 09:55:11.295682 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.295654    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:11.398100 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.398070    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-lnql9"]
Jul 03 09:55:11.400503 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.400484    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lnql9"
Jul 03 09:55:11.411702 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.411676    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lnql9"]
Jul 03 09:55:11.485069 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.485041    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zh8lr"
Jul 03 09:55:11.486934 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.486905    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqcmg\" (UniqueName: \"kubernetes.io/projected/df2b0e93-80a3-475a-ac4a-69f8e3f2a95b-kube-api-access-bqcmg\") pod \"authorino-75d556bd77-lnql9\" (UID: \"df2b0e93-80a3-475a-ac4a-69f8e3f2a95b\") " pod="kuadrant-system/authorino-75d556bd77-lnql9"
Jul 03 09:55:11.587946 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.587915    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqcmg\" (UniqueName: \"kubernetes.io/projected/df2b0e93-80a3-475a-ac4a-69f8e3f2a95b-kube-api-access-bqcmg\") pod \"authorino-75d556bd77-lnql9\" (UID: \"df2b0e93-80a3-475a-ac4a-69f8e3f2a95b\") " pod="kuadrant-system/authorino-75d556bd77-lnql9"
Jul 03 09:55:11.597795 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.597629    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqcmg\" (UniqueName: \"kubernetes.io/projected/df2b0e93-80a3-475a-ac4a-69f8e3f2a95b-kube-api-access-bqcmg\") pod \"authorino-75d556bd77-lnql9\" (UID: \"df2b0e93-80a3-475a-ac4a-69f8e3f2a95b\") " pod="kuadrant-system/authorino-75d556bd77-lnql9"
Jul 03 09:55:11.640671 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.640642    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-zh8lr"]
Jul 03 09:55:11.643109 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:55:11.643079    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f06c156_5631_446c_a4d0_ab01adf84c5b.slice/crio-67d3dd39636aefbda2c5b9ff8ac7fc4643e6e3110a19012ad44b5d4751245077 WatchSource:0}: Error finding container 67d3dd39636aefbda2c5b9ff8ac7fc4643e6e3110a19012ad44b5d4751245077: Status 404 returned error can't find the container with id 67d3dd39636aefbda2c5b9ff8ac7fc4643e6e3110a19012ad44b5d4751245077
Jul 03 09:55:11.682045 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.681957    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lnjn" event={"ID":"84b5540a-d878-43fd-9154-3d431171bf7c","Type":"ContainerStarted","Data":"3d249f763fe84fed5dbb3ef492a069bd9012a6ff0a562d7f0f427570eed1c4bc"}
Jul 03 09:55:11.684100 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.684069    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zh8lr" event={"ID":"9f06c156-5631-446c-a4d0-ab01adf84c5b","Type":"ContainerStarted","Data":"67d3dd39636aefbda2c5b9ff8ac7fc4643e6e3110a19012ad44b5d4751245077"}
Jul 03 09:55:11.709951 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.709914    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lnql9"
Jul 03 09:55:11.924232 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:11.923600    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lnql9"]
Jul 03 09:55:11.927153 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:55:11.927120    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf2b0e93_80a3_475a_ac4a_69f8e3f2a95b.slice/crio-77a32e763c1e9452d8400a258bdf83e6cb9a5673afacda3076f2b1a19fa06d9a WatchSource:0}: Error finding container 77a32e763c1e9452d8400a258bdf83e6cb9a5673afacda3076f2b1a19fa06d9a: Status 404 returned error can't find the container with id 77a32e763c1e9452d8400a258bdf83e6cb9a5673afacda3076f2b1a19fa06d9a
Jul 03 09:55:12.695177 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:12.695102    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lnql9" event={"ID":"df2b0e93-80a3-475a-ac4a-69f8e3f2a95b","Type":"ContainerStarted","Data":"77a32e763c1e9452d8400a258bdf83e6cb9a5673afacda3076f2b1a19fa06d9a"}
Jul 03 09:55:16.721628 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:16.721586    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zh8lr" event={"ID":"9f06c156-5631-446c-a4d0-ab01adf84c5b","Type":"ContainerStarted","Data":"d05f877f5f533f588ec53d4d11ba5581e503b3bdc42051f196a315e08f3bee64"}
Jul 03 09:55:16.723151 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:16.723120    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lnql9" event={"ID":"df2b0e93-80a3-475a-ac4a-69f8e3f2a95b","Type":"ContainerStarted","Data":"2f084ccf81c4bee38cf8ae2f2f37c06ba28efb161167549795f561102e262b54"}
Jul 03 09:55:16.724556 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:16.724530    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lnjn" event={"ID":"84b5540a-d878-43fd-9154-3d431171bf7c","Type":"ContainerStarted","Data":"0bf0aba34b60149d12a045179d9de0048630ae050a28631c05c2196d8190e2f2"}
Jul 03 09:55:16.724691 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:16.724601    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"
Jul 03 09:55:16.737712 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:16.737666    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-zh8lr" podStartSLOduration=1.362381171 podStartE2EDuration="5.737651736s" totalImagesPullingTime="4.375270565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:55:16.735688901 +0000 UTC m=+718.600977556" watchObservedRunningTime="2026-07-03 09:55:16.737651736 +0000 UTC m=+718.602940390"
Jul 03 09:55:16.754077 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:16.753791    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lnjn" podStartSLOduration=1.8520677079999999 podStartE2EDuration="6.753771585s" totalImagesPullingTime="4.901703877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:55:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:55:16.752027244 +0000 UTC m=+718.617315899" watchObservedRunningTime="2026-07-03 09:55:16.753771585 +0000 UTC m=+718.619060241"
Jul 03 09:55:16.766770 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:16.766716    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-lnql9" podStartSLOduration=1.674987562 podStartE2EDuration="5.76670027s" totalImagesPullingTime="4.091712708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:55:16.766014752 +0000 UTC m=+718.631303424" watchObservedRunningTime="2026-07-03 09:55:16.76670027 +0000 UTC m=+718.631988925"
Jul 03 09:55:16.798694 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:16.798657    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-zh8lr"]
Jul 03 09:55:18.734318 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:18.734280    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-zh8lr" podUID="9f06c156-5631-446c-a4d0-ab01adf84c5b" containerName="authorino" containerID="cri-o://d05f877f5f533f588ec53d4d11ba5581e503b3bdc42051f196a315e08f3bee64" gracePeriod=30
Jul 03 09:55:18.977583 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:18.977558    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zh8lr"
Jul 03 09:55:19.052920 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:19.052885    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbk2q\" (UniqueName: \"kubernetes.io/projected/9f06c156-5631-446c-a4d0-ab01adf84c5b-kube-api-access-zbk2q\") pod \"9f06c156-5631-446c-a4d0-ab01adf84c5b\" (UID: \"9f06c156-5631-446c-a4d0-ab01adf84c5b\") "
Jul 03 09:55:19.054933 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:19.054905    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f06c156-5631-446c-a4d0-ab01adf84c5b-kube-api-access-zbk2q" (OuterVolumeSpecName: "kube-api-access-zbk2q") pod "9f06c156-5631-446c-a4d0-ab01adf84c5b" (UID: "9f06c156-5631-446c-a4d0-ab01adf84c5b"). InnerVolumeSpecName "kube-api-access-zbk2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:55:19.153501 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:19.153463    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbk2q\" (UniqueName: \"kubernetes.io/projected/9f06c156-5631-446c-a4d0-ab01adf84c5b-kube-api-access-zbk2q\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:55:19.740541 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:19.740506    2571 generic.go:356] "Generic (PLEG): container finished" podID="9f06c156-5631-446c-a4d0-ab01adf84c5b" containerID="d05f877f5f533f588ec53d4d11ba5581e503b3bdc42051f196a315e08f3bee64" exitCode=0
Jul 03 09:55:19.740970 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:19.740572    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zh8lr" event={"ID":"9f06c156-5631-446c-a4d0-ab01adf84c5b","Type":"ContainerDied","Data":"d05f877f5f533f588ec53d4d11ba5581e503b3bdc42051f196a315e08f3bee64"}
Jul 03 09:55:19.740970 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:19.740615    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zh8lr" event={"ID":"9f06c156-5631-446c-a4d0-ab01adf84c5b","Type":"ContainerDied","Data":"67d3dd39636aefbda2c5b9ff8ac7fc4643e6e3110a19012ad44b5d4751245077"}
Jul 03 09:55:19.740970 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:19.740632    2571 scope.go:117] "RemoveContainer" containerID="d05f877f5f533f588ec53d4d11ba5581e503b3bdc42051f196a315e08f3bee64"
Jul 03 09:55:19.740970 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:19.740637    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zh8lr"
Jul 03 09:55:19.749962 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:19.749947    2571 scope.go:117] "RemoveContainer" containerID="d05f877f5f533f588ec53d4d11ba5581e503b3bdc42051f196a315e08f3bee64"
Jul 03 09:55:19.750248 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:55:19.750229    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d05f877f5f533f588ec53d4d11ba5581e503b3bdc42051f196a315e08f3bee64\": container with ID starting with d05f877f5f533f588ec53d4d11ba5581e503b3bdc42051f196a315e08f3bee64 not found: ID does not exist" containerID="d05f877f5f533f588ec53d4d11ba5581e503b3bdc42051f196a315e08f3bee64"
Jul 03 09:55:19.750297 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:19.750258    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d05f877f5f533f588ec53d4d11ba5581e503b3bdc42051f196a315e08f3bee64"} err="failed to get container status \"d05f877f5f533f588ec53d4d11ba5581e503b3bdc42051f196a315e08f3bee64\": rpc error: code = NotFound desc = could not find container \"d05f877f5f533f588ec53d4d11ba5581e503b3bdc42051f196a315e08f3bee64\": container with ID starting with d05f877f5f533f588ec53d4d11ba5581e503b3bdc42051f196a315e08f3bee64 not found: ID does not exist"
Jul 03 09:55:19.763218 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:19.763193    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-zh8lr"]
Jul 03 09:55:19.768206 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:19.768182    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-zh8lr"]
Jul 03 09:55:20.764988 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:20.764959    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f06c156-5631-446c-a4d0-ab01adf84c5b" path="/var/lib/kubelet/pods/9f06c156-5631-446c-a4d0-ab01adf84c5b/volumes"
Jul 03 09:55:27.732384 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:27.732358    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"
Jul 03 09:55:31.979577 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:31.979543    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:43.752249 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:43.752205    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:43.979756 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:43.979723    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-q9k77"]
Jul 03 09:55:43.980152 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:43.980138    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f06c156-5631-446c-a4d0-ab01adf84c5b" containerName="authorino"
Jul 03 09:55:43.980208 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:43.980154    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f06c156-5631-446c-a4d0-ab01adf84c5b" containerName="authorino"
Jul 03 09:55:43.980245 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:43.980215    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f06c156-5631-446c-a4d0-ab01adf84c5b" containerName="authorino"
Jul 03 09:55:43.983484 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:43.983453    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-q9k77"
Jul 03 09:55:43.992336 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:43.992311    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-q9k77"]
Jul 03 09:55:44.054169 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.054089    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:44.083570 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.083536    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjwch\" (UniqueName: \"kubernetes.io/projected/593f070d-7896-42c7-bbf0-1af709c8fec5-kube-api-access-zjwch\") pod \"authorino-5d4b67c446-q9k77\" (UID: \"593f070d-7896-42c7-bbf0-1af709c8fec5\") " pod="kuadrant-system/authorino-5d4b67c446-q9k77"
Jul 03 09:55:44.184667 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.184631    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjwch\" (UniqueName: \"kubernetes.io/projected/593f070d-7896-42c7-bbf0-1af709c8fec5-kube-api-access-zjwch\") pod \"authorino-5d4b67c446-q9k77\" (UID: \"593f070d-7896-42c7-bbf0-1af709c8fec5\") " pod="kuadrant-system/authorino-5d4b67c446-q9k77"
Jul 03 09:55:44.198674 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.198637    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjwch\" (UniqueName: \"kubernetes.io/projected/593f070d-7896-42c7-bbf0-1af709c8fec5-kube-api-access-zjwch\") pod \"authorino-5d4b67c446-q9k77\" (UID: \"593f070d-7896-42c7-bbf0-1af709c8fec5\") " pod="kuadrant-system/authorino-5d4b67c446-q9k77"
Jul 03 09:55:44.211310 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.211279    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-q9k77"]
Jul 03 09:55:44.211535 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.211524    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-q9k77"
Jul 03 09:55:44.248064 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.247805    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cb786cbb-p6p8m"]
Jul 03 09:55:44.254219 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.254188    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb786cbb-p6p8m"
Jul 03 09:55:44.259143 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.259095    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cb786cbb-p6p8m"]
Jul 03 09:55:44.287348 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.287273    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cb786cbb-p6p8m"]
Jul 03 09:55:44.287669 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:55:44.287611    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-w8vx8], unattached volumes=[], failed to process volumes=[kube-api-access-w8vx8]: context canceled" pod="kuadrant-system/authorino-cb786cbb-p6p8m" podUID="c29c5736-f6cc-4a4d-b466-e4ed90ae6e61"
Jul 03 09:55:44.318575 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.318537    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6944578694-ss64h"]
Jul 03 09:55:44.323457 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.323435    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6944578694-ss64h"
Jul 03 09:55:44.326133 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.325911    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 03 09:55:44.330035 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.330005    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6944578694-ss64h"]
Jul 03 09:55:44.361970 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.361948    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-q9k77"]
Jul 03 09:55:44.364173 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:55:44.364139    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod593f070d_7896_42c7_bbf0_1af709c8fec5.slice/crio-3161753410b4f29724e5698a9f5e734d7345fc27edf9aaaf67688c23665a1a27 WatchSource:0}: Error finding container 3161753410b4f29724e5698a9f5e734d7345fc27edf9aaaf67688c23665a1a27: Status 404 returned error can't find the container with id 3161753410b4f29724e5698a9f5e734d7345fc27edf9aaaf67688c23665a1a27
Jul 03 09:55:44.387202 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.387171    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8vx8\" (UniqueName: \"kubernetes.io/projected/c29c5736-f6cc-4a4d-b466-e4ed90ae6e61-kube-api-access-w8vx8\") pod \"authorino-cb786cbb-p6p8m\" (UID: \"c29c5736-f6cc-4a4d-b466-e4ed90ae6e61\") " pod="kuadrant-system/authorino-cb786cbb-p6p8m"
Jul 03 09:55:44.488069 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.488031    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad173716-8a45-4444-af9d-1258893c77b5-tls-cert\") pod \"authorino-6944578694-ss64h\" (UID: \"ad173716-8a45-4444-af9d-1258893c77b5\") " pod="kuadrant-system/authorino-6944578694-ss64h"
Jul 03 09:55:44.488069 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.488066    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f4s9\" (UniqueName: \"kubernetes.io/projected/ad173716-8a45-4444-af9d-1258893c77b5-kube-api-access-6f4s9\") pod \"authorino-6944578694-ss64h\" (UID: \"ad173716-8a45-4444-af9d-1258893c77b5\") " pod="kuadrant-system/authorino-6944578694-ss64h"
Jul 03 09:55:44.488293 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.488089    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8vx8\" (UniqueName: \"kubernetes.io/projected/c29c5736-f6cc-4a4d-b466-e4ed90ae6e61-kube-api-access-w8vx8\") pod \"authorino-cb786cbb-p6p8m\" (UID: \"c29c5736-f6cc-4a4d-b466-e4ed90ae6e61\") " pod="kuadrant-system/authorino-cb786cbb-p6p8m"
Jul 03 09:55:44.498219 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.498194    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8vx8\" (UniqueName: \"kubernetes.io/projected/c29c5736-f6cc-4a4d-b466-e4ed90ae6e61-kube-api-access-w8vx8\") pod \"authorino-cb786cbb-p6p8m\" (UID: \"c29c5736-f6cc-4a4d-b466-e4ed90ae6e61\") " pod="kuadrant-system/authorino-cb786cbb-p6p8m"
Jul 03 09:55:44.588936 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.588819    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad173716-8a45-4444-af9d-1258893c77b5-tls-cert\") pod \"authorino-6944578694-ss64h\" (UID: \"ad173716-8a45-4444-af9d-1258893c77b5\") " pod="kuadrant-system/authorino-6944578694-ss64h"
Jul 03 09:55:44.588936 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.588880    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f4s9\" (UniqueName: \"kubernetes.io/projected/ad173716-8a45-4444-af9d-1258893c77b5-kube-api-access-6f4s9\") pod \"authorino-6944578694-ss64h\" (UID: \"ad173716-8a45-4444-af9d-1258893c77b5\") " pod="kuadrant-system/authorino-6944578694-ss64h"
Jul 03 09:55:44.591244 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.591221    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad173716-8a45-4444-af9d-1258893c77b5-tls-cert\") pod \"authorino-6944578694-ss64h\" (UID: \"ad173716-8a45-4444-af9d-1258893c77b5\") " pod="kuadrant-system/authorino-6944578694-ss64h"
Jul 03 09:55:44.597913 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.597890    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f4s9\" (UniqueName: \"kubernetes.io/projected/ad173716-8a45-4444-af9d-1258893c77b5-kube-api-access-6f4s9\") pod \"authorino-6944578694-ss64h\" (UID: \"ad173716-8a45-4444-af9d-1258893c77b5\") " pod="kuadrant-system/authorino-6944578694-ss64h"
Jul 03 09:55:44.635894 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.635863    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6944578694-ss64h"
Jul 03 09:55:44.778027 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.778002    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6944578694-ss64h"]
Jul 03 09:55:44.866769 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.866736    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6944578694-ss64h" event={"ID":"ad173716-8a45-4444-af9d-1258893c77b5","Type":"ContainerStarted","Data":"068e5fc9f49b21ee993f18c9107293f77e53216815a00741ab8ac0e79002ecec"}
Jul 03 09:55:44.868279 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.868253    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb786cbb-p6p8m"
Jul 03 09:55:44.868410 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.868272    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-q9k77" event={"ID":"593f070d-7896-42c7-bbf0-1af709c8fec5","Type":"ContainerStarted","Data":"52511ba79f53b82181a8c5cb0a511fad1d52441a5d93f86f069f2e47e4322cb6"}
Jul 03 09:55:44.868410 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.868310    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-q9k77" event={"ID":"593f070d-7896-42c7-bbf0-1af709c8fec5","Type":"ContainerStarted","Data":"3161753410b4f29724e5698a9f5e734d7345fc27edf9aaaf67688c23665a1a27"}
Jul 03 09:55:44.868410 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.868316    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-q9k77" podUID="593f070d-7896-42c7-bbf0-1af709c8fec5" containerName="authorino" containerID="cri-o://52511ba79f53b82181a8c5cb0a511fad1d52441a5d93f86f069f2e47e4322cb6" gracePeriod=30
Jul 03 09:55:44.875591 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.875573    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb786cbb-p6p8m"
Jul 03 09:55:44.900819 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.900767    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-q9k77" podStartSLOduration=1.475245591 podStartE2EDuration="1.900751201s" totalImagesPullingTime="425.50561ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:55:44.89988505 +0000 UTC m=+746.765173704" watchObservedRunningTime="2026-07-03 09:55:44.900751201 +0000 UTC m=+746.766039889"
Jul 03 09:55:44.992279 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.992251    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8vx8\" (UniqueName: \"kubernetes.io/projected/c29c5736-f6cc-4a4d-b466-e4ed90ae6e61-kube-api-access-w8vx8\") pod \"c29c5736-f6cc-4a4d-b466-e4ed90ae6e61\" (UID: \"c29c5736-f6cc-4a4d-b466-e4ed90ae6e61\") "
Jul 03 09:55:44.994215 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:44.994190    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c29c5736-f6cc-4a4d-b466-e4ed90ae6e61-kube-api-access-w8vx8" (OuterVolumeSpecName: "kube-api-access-w8vx8") pod "c29c5736-f6cc-4a4d-b466-e4ed90ae6e61" (UID: "c29c5736-f6cc-4a4d-b466-e4ed90ae6e61"). InnerVolumeSpecName "kube-api-access-w8vx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:55:45.093298 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.093264    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8vx8\" (UniqueName: \"kubernetes.io/projected/c29c5736-f6cc-4a4d-b466-e4ed90ae6e61-kube-api-access-w8vx8\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:55:45.110460 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.110426    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-q9k77"
Jul 03 09:55:45.193657 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.193637    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjwch\" (UniqueName: \"kubernetes.io/projected/593f070d-7896-42c7-bbf0-1af709c8fec5-kube-api-access-zjwch\") pod \"593f070d-7896-42c7-bbf0-1af709c8fec5\" (UID: \"593f070d-7896-42c7-bbf0-1af709c8fec5\") "
Jul 03 09:55:45.195410 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.195383    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/593f070d-7896-42c7-bbf0-1af709c8fec5-kube-api-access-zjwch" (OuterVolumeSpecName: "kube-api-access-zjwch") pod "593f070d-7896-42c7-bbf0-1af709c8fec5" (UID: "593f070d-7896-42c7-bbf0-1af709c8fec5"). InnerVolumeSpecName "kube-api-access-zjwch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:55:45.294725 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.294695    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjwch\" (UniqueName: \"kubernetes.io/projected/593f070d-7896-42c7-bbf0-1af709c8fec5-kube-api-access-zjwch\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:55:45.875016 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.874979    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6944578694-ss64h" event={"ID":"ad173716-8a45-4444-af9d-1258893c77b5","Type":"ContainerStarted","Data":"949e2028ae9588480c3426ee0737e97f8d5b6fc6b4b980dad5e5632402140ef2"}
Jul 03 09:55:45.876241 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.876221    2571 generic.go:356] "Generic (PLEG): container finished" podID="593f070d-7896-42c7-bbf0-1af709c8fec5" containerID="52511ba79f53b82181a8c5cb0a511fad1d52441a5d93f86f069f2e47e4322cb6" exitCode=2
Jul 03 09:55:45.876318 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.876250    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-q9k77"
Jul 03 09:55:45.876357 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.876321    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb786cbb-p6p8m"
Jul 03 09:55:45.876357 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.876341    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-q9k77" event={"ID":"593f070d-7896-42c7-bbf0-1af709c8fec5","Type":"ContainerDied","Data":"52511ba79f53b82181a8c5cb0a511fad1d52441a5d93f86f069f2e47e4322cb6"}
Jul 03 09:55:45.876427 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.876377    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-q9k77" event={"ID":"593f070d-7896-42c7-bbf0-1af709c8fec5","Type":"ContainerDied","Data":"3161753410b4f29724e5698a9f5e734d7345fc27edf9aaaf67688c23665a1a27"}
Jul 03 09:55:45.876427 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.876396    2571 scope.go:117] "RemoveContainer" containerID="52511ba79f53b82181a8c5cb0a511fad1d52441a5d93f86f069f2e47e4322cb6"
Jul 03 09:55:45.886198 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.886182    2571 scope.go:117] "RemoveContainer" containerID="52511ba79f53b82181a8c5cb0a511fad1d52441a5d93f86f069f2e47e4322cb6"
Jul 03 09:55:45.886448 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:55:45.886428    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52511ba79f53b82181a8c5cb0a511fad1d52441a5d93f86f069f2e47e4322cb6\": container with ID starting with 52511ba79f53b82181a8c5cb0a511fad1d52441a5d93f86f069f2e47e4322cb6 not found: ID does not exist" containerID="52511ba79f53b82181a8c5cb0a511fad1d52441a5d93f86f069f2e47e4322cb6"
Jul 03 09:55:45.886507 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.886459    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52511ba79f53b82181a8c5cb0a511fad1d52441a5d93f86f069f2e47e4322cb6"} err="failed to get container status \"52511ba79f53b82181a8c5cb0a511fad1d52441a5d93f86f069f2e47e4322cb6\": rpc error: code = NotFound desc = could not find container \"52511ba79f53b82181a8c5cb0a511fad1d52441a5d93f86f069f2e47e4322cb6\": container with ID starting with 52511ba79f53b82181a8c5cb0a511fad1d52441a5d93f86f069f2e47e4322cb6 not found: ID does not exist"
Jul 03 09:55:45.906882 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.906807    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6944578694-ss64h" podStartSLOduration=1.508442571 podStartE2EDuration="1.906791663s" totalImagesPullingTime="398.349092ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:55:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:55:45.906204872 +0000 UTC m=+747.771493525" watchObservedRunningTime="2026-07-03 09:55:45.906791663 +0000 UTC m=+747.772080318"
Jul 03 09:55:45.948894 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.948846    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cb786cbb-p6p8m"]
Jul 03 09:55:45.951115 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.951087    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lnql9"]
Jul 03 09:55:45.951342 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.951306    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-lnql9" podUID="df2b0e93-80a3-475a-ac4a-69f8e3f2a95b" containerName="authorino" containerID="cri-o://2f084ccf81c4bee38cf8ae2f2f37c06ba28efb161167549795f561102e262b54" gracePeriod=30
Jul 03 09:55:45.969209 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.969175    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-cb786cbb-p6p8m"]
Jul 03 09:55:45.999554 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:45.999522    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-q9k77"]
Jul 03 09:55:46.011901 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.011872    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-q9k77"]
Jul 03 09:55:46.213852 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.213808    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lnql9"
Jul 03 09:55:46.304701 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.304673    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqcmg\" (UniqueName: \"kubernetes.io/projected/df2b0e93-80a3-475a-ac4a-69f8e3f2a95b-kube-api-access-bqcmg\") pod \"df2b0e93-80a3-475a-ac4a-69f8e3f2a95b\" (UID: \"df2b0e93-80a3-475a-ac4a-69f8e3f2a95b\") "
Jul 03 09:55:46.306730 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.306706    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df2b0e93-80a3-475a-ac4a-69f8e3f2a95b-kube-api-access-bqcmg" (OuterVolumeSpecName: "kube-api-access-bqcmg") pod "df2b0e93-80a3-475a-ac4a-69f8e3f2a95b" (UID: "df2b0e93-80a3-475a-ac4a-69f8e3f2a95b"). InnerVolumeSpecName "kube-api-access-bqcmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:55:46.406306 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.406270    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqcmg\" (UniqueName: \"kubernetes.io/projected/df2b0e93-80a3-475a-ac4a-69f8e3f2a95b-kube-api-access-bqcmg\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:55:46.764971 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.764942    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="593f070d-7896-42c7-bbf0-1af709c8fec5" path="/var/lib/kubelet/pods/593f070d-7896-42c7-bbf0-1af709c8fec5/volumes"
Jul 03 09:55:46.765275 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.765262    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c29c5736-f6cc-4a4d-b466-e4ed90ae6e61" path="/var/lib/kubelet/pods/c29c5736-f6cc-4a4d-b466-e4ed90ae6e61/volumes"
Jul 03 09:55:46.882363 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.882331    2571 generic.go:356] "Generic (PLEG): container finished" podID="df2b0e93-80a3-475a-ac4a-69f8e3f2a95b" containerID="2f084ccf81c4bee38cf8ae2f2f37c06ba28efb161167549795f561102e262b54" exitCode=0
Jul 03 09:55:46.882791 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.882370    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lnql9"
Jul 03 09:55:46.882791 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.882387    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lnql9" event={"ID":"df2b0e93-80a3-475a-ac4a-69f8e3f2a95b","Type":"ContainerDied","Data":"2f084ccf81c4bee38cf8ae2f2f37c06ba28efb161167549795f561102e262b54"}
Jul 03 09:55:46.882791 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.882431    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lnql9" event={"ID":"df2b0e93-80a3-475a-ac4a-69f8e3f2a95b","Type":"ContainerDied","Data":"77a32e763c1e9452d8400a258bdf83e6cb9a5673afacda3076f2b1a19fa06d9a"}
Jul 03 09:55:46.882791 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.882456    2571 scope.go:117] "RemoveContainer" containerID="2f084ccf81c4bee38cf8ae2f2f37c06ba28efb161167549795f561102e262b54"
Jul 03 09:55:46.891086 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.891069    2571 scope.go:117] "RemoveContainer" containerID="2f084ccf81c4bee38cf8ae2f2f37c06ba28efb161167549795f561102e262b54"
Jul 03 09:55:46.891357 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:55:46.891339    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f084ccf81c4bee38cf8ae2f2f37c06ba28efb161167549795f561102e262b54\": container with ID starting with 2f084ccf81c4bee38cf8ae2f2f37c06ba28efb161167549795f561102e262b54 not found: ID does not exist" containerID="2f084ccf81c4bee38cf8ae2f2f37c06ba28efb161167549795f561102e262b54"
Jul 03 09:55:46.891398 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.891364    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f084ccf81c4bee38cf8ae2f2f37c06ba28efb161167549795f561102e262b54"} err="failed to get container status \"2f084ccf81c4bee38cf8ae2f2f37c06ba28efb161167549795f561102e262b54\": rpc error: code = NotFound desc = could not find container \"2f084ccf81c4bee38cf8ae2f2f37c06ba28efb161167549795f561102e262b54\": container with ID starting with 2f084ccf81c4bee38cf8ae2f2f37c06ba28efb161167549795f561102e262b54 not found: ID does not exist"
Jul 03 09:55:46.899557 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.899525    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lnql9"]
Jul 03 09:55:46.906029 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:46.906006    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lnql9"]
Jul 03 09:55:48.766285 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:48.766253    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df2b0e93-80a3-475a-ac4a-69f8e3f2a95b" path="/var/lib/kubelet/pods/df2b0e93-80a3-475a-ac4a-69f8e3f2a95b/volumes"
Jul 03 09:55:50.355205 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:50.355173    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:50.437094 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:50.437061    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:50.509715 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:50.509682    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:50.585373 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:50.585336    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:53.153326 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:53.153292    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:55:53.251299 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:55:53.251261    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:56:25.040455 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:56:25.040417    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:56:25.434584 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:56:25.434547    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:56:25.530175 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:56:25.530145    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:56:25.868620 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:56:25.868540    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:56:25.969614 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:56:25.969580    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:56:26.090226 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:56:26.090191    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:56:26.647066 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:56:26.647033    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:56:26.749601 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:56:26.749562    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:56:36.599774 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:56:36.599731    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:56:36.699372 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:56:36.699341    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:56:49.759130 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:56:49.759091    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:56:50.200595 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:56:50.200554    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:57:00.699314 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.699281    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"]
Jul 03 09:57:00.699792 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.699758    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="593f070d-7896-42c7-bbf0-1af709c8fec5" containerName="authorino"
Jul 03 09:57:00.699792 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.699776    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="593f070d-7896-42c7-bbf0-1af709c8fec5" containerName="authorino"
Jul 03 09:57:00.699792 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.699790    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df2b0e93-80a3-475a-ac4a-69f8e3f2a95b" containerName="authorino"
Jul 03 09:57:00.700031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.699799    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="df2b0e93-80a3-475a-ac4a-69f8e3f2a95b" containerName="authorino"
Jul 03 09:57:00.700031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.699924    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="df2b0e93-80a3-475a-ac4a-69f8e3f2a95b" containerName="authorino"
Jul 03 09:57:00.700031 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.699935    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="593f070d-7896-42c7-bbf0-1af709c8fec5" containerName="authorino"
Jul 03 09:57:00.703193 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.703169    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.705729 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.705708    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 03 09:57:00.705871 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.705815    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m2wrc\""
Jul 03 09:57:00.707715 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.707694    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 03 09:57:00.707809 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.707721    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 03 09:57:00.715337 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.715308    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"]
Jul 03 09:57:00.738495 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.738462    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:57:00.791908 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.791878    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a8b170b-8cdb-44a2-94dc-833e5450faf7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.791908 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.791915    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a8b170b-8cdb-44a2-94dc-833e5450faf7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.792122 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.791935    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a8b170b-8cdb-44a2-94dc-833e5450faf7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.792122 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.792011    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a8b170b-8cdb-44a2-94dc-833e5450faf7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.792122 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.792073    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgcp9\" (UniqueName: \"kubernetes.io/projected/8a8b170b-8cdb-44a2-94dc-833e5450faf7-kube-api-access-hgcp9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.792232 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.792127    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a8b170b-8cdb-44a2-94dc-833e5450faf7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.892589 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.892540    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgcp9\" (UniqueName: \"kubernetes.io/projected/8a8b170b-8cdb-44a2-94dc-833e5450faf7-kube-api-access-hgcp9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.892589 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.892581    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a8b170b-8cdb-44a2-94dc-833e5450faf7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.892881 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.892648    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a8b170b-8cdb-44a2-94dc-833e5450faf7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.892881 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.892674    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a8b170b-8cdb-44a2-94dc-833e5450faf7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.892881 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.892701    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a8b170b-8cdb-44a2-94dc-833e5450faf7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.892881 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.892750    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a8b170b-8cdb-44a2-94dc-833e5450faf7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.893249 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.893206    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a8b170b-8cdb-44a2-94dc-833e5450faf7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.893249 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.893228    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a8b170b-8cdb-44a2-94dc-833e5450faf7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.893401 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.893255    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a8b170b-8cdb-44a2-94dc-833e5450faf7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.895125 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.895104    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a8b170b-8cdb-44a2-94dc-833e5450faf7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.895300 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.895284    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a8b170b-8cdb-44a2-94dc-833e5450faf7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:00.901056 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:00.901037    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgcp9\" (UniqueName: \"kubernetes.io/projected/8a8b170b-8cdb-44a2-94dc-833e5450faf7-kube-api-access-hgcp9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n5rq2\" (UID: \"8a8b170b-8cdb-44a2-94dc-833e5450faf7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:01.015665 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:01.015569    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:01.160632 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:01.160468    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"]
Jul 03 09:57:01.163084 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:57:01.163054    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a8b170b_8cdb_44a2_94dc_833e5450faf7.slice/crio-bd3cf4ba8f62d35752a9f3b8d494444506fe64e231152bdc18a92afc3eb3421c WatchSource:0}: Error finding container bd3cf4ba8f62d35752a9f3b8d494444506fe64e231152bdc18a92afc3eb3421c: Status 404 returned error can't find the container with id bd3cf4ba8f62d35752a9f3b8d494444506fe64e231152bdc18a92afc3eb3421c
Jul 03 09:57:01.262990 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:01.262950    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2" event={"ID":"8a8b170b-8cdb-44a2-94dc-833e5450faf7","Type":"ContainerStarted","Data":"bd3cf4ba8f62d35752a9f3b8d494444506fe64e231152bdc18a92afc3eb3421c"}
Jul 03 09:57:01.515254 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:01.515210    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:57:03.278108 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:03.278070    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2" event={"ID":"8a8b170b-8cdb-44a2-94dc-833e5450faf7","Type":"ContainerStarted","Data":"872dac30641bb531b7ec006089791332412d8bd726fa7a1a6115d4a42c74a5a6"}
Jul 03 09:57:03.297297 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:03.297229    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2" podStartSLOduration=2.17320836 podStartE2EDuration="3.297209132s" totalImagesPullingTime="1.124000772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:57:03.295023656 +0000 UTC m=+825.160312312" watchObservedRunningTime="2026-07-03 09:57:03.297209132 +0000 UTC m=+825.162497765"
Jul 03 09:57:06.539097 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:06.539059    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:57:07.753046 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:07.753007    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:57:11.015807 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:11.015773    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:11.016316 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:11.015875    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:11.027679 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:11.027656    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:11.329773 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:11.329690    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n5rq2"
Jul 03 09:57:31.881385 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:31.881344    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:57:33.469067 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:33.469028    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:57:49.340952 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:49.340902    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:57:51.211942 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:57:51.211908    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:58:00.144438 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:00.144404    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:58:02.544664 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:02.544610    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:58:15.329294 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:15.329206    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:58:16.303103 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:16.303068    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:58:18.290760 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:18.290731    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:58:18.660169 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:18.660141    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 09:58:18.663513 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:18.663489    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 09:58:52.961071 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:52.961003    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6cc8dc5447-m97dg"]
Jul 03 09:58:52.964356 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:52.964340    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cc8dc5447-m97dg"
Jul 03 09:58:52.971812 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:52.971785    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cc8dc5447-m97dg"]
Jul 03 09:58:53.091654 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:53.091619    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv9bk\" (UniqueName: \"kubernetes.io/projected/ec6aae00-836b-4613-a5a5-a0a5b91c4edf-kube-api-access-gv9bk\") pod \"authorino-6cc8dc5447-m97dg\" (UID: \"ec6aae00-836b-4613-a5a5-a0a5b91c4edf\") " pod="kuadrant-system/authorino-6cc8dc5447-m97dg"
Jul 03 09:58:53.091872 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:53.091670    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec6aae00-836b-4613-a5a5-a0a5b91c4edf-tls-cert\") pod \"authorino-6cc8dc5447-m97dg\" (UID: \"ec6aae00-836b-4613-a5a5-a0a5b91c4edf\") " pod="kuadrant-system/authorino-6cc8dc5447-m97dg"
Jul 03 09:58:53.192883 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:53.192828    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv9bk\" (UniqueName: \"kubernetes.io/projected/ec6aae00-836b-4613-a5a5-a0a5b91c4edf-kube-api-access-gv9bk\") pod \"authorino-6cc8dc5447-m97dg\" (UID: \"ec6aae00-836b-4613-a5a5-a0a5b91c4edf\") " pod="kuadrant-system/authorino-6cc8dc5447-m97dg"
Jul 03 09:58:53.193053 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:53.192896    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec6aae00-836b-4613-a5a5-a0a5b91c4edf-tls-cert\") pod \"authorino-6cc8dc5447-m97dg\" (UID: \"ec6aae00-836b-4613-a5a5-a0a5b91c4edf\") " pod="kuadrant-system/authorino-6cc8dc5447-m97dg"
Jul 03 09:58:53.195294 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:53.195274    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec6aae00-836b-4613-a5a5-a0a5b91c4edf-tls-cert\") pod \"authorino-6cc8dc5447-m97dg\" (UID: \"ec6aae00-836b-4613-a5a5-a0a5b91c4edf\") " pod="kuadrant-system/authorino-6cc8dc5447-m97dg"
Jul 03 09:58:53.201229 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:53.201204    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv9bk\" (UniqueName: \"kubernetes.io/projected/ec6aae00-836b-4613-a5a5-a0a5b91c4edf-kube-api-access-gv9bk\") pod \"authorino-6cc8dc5447-m97dg\" (UID: \"ec6aae00-836b-4613-a5a5-a0a5b91c4edf\") " pod="kuadrant-system/authorino-6cc8dc5447-m97dg"
Jul 03 09:58:53.274652 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:53.274569    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cc8dc5447-m97dg"
Jul 03 09:58:53.343711 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:53.343653    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:58:53.425521 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:53.425480    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cc8dc5447-m97dg"]
Jul 03 09:58:53.427466 ip-10-0-138-129 kubenswrapper[2571]: W0703 09:58:53.427441    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec6aae00_836b_4613_a5a5_a0a5b91c4edf.slice/crio-7fa35cf5c9313628fa5e4c1a166e91027cab3795857a4d943383ec6e76840a33 WatchSource:0}: Error finding container 7fa35cf5c9313628fa5e4c1a166e91027cab3795857a4d943383ec6e76840a33: Status 404 returned error can't find the container with id 7fa35cf5c9313628fa5e4c1a166e91027cab3795857a4d943383ec6e76840a33
Jul 03 09:58:53.428806 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:53.428785    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 09:58:53.824069 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:53.824041    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cc8dc5447-m97dg" event={"ID":"ec6aae00-836b-4613-a5a5-a0a5b91c4edf","Type":"ContainerStarted","Data":"7fa35cf5c9313628fa5e4c1a166e91027cab3795857a4d943383ec6e76840a33"}
Jul 03 09:58:54.624075 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:54.624029    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:58:54.833182 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:54.833125    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cc8dc5447-m97dg" event={"ID":"ec6aae00-836b-4613-a5a5-a0a5b91c4edf","Type":"ContainerStarted","Data":"23d21f945002f99650f5058e0f0bded1a3a5fe59d749a291e1ecab37ded4c00b"}
Jul 03 09:58:54.850168 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:54.850095    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6cc8dc5447-m97dg" podStartSLOduration=2.470772364 podStartE2EDuration="2.850071091s" totalImagesPullingTime="379.298727ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 09:58:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 09:58:54.849476628 +0000 UTC m=+936.714765284" watchObservedRunningTime="2026-07-03 09:58:54.850071091 +0000 UTC m=+936.715359738"
Jul 03 09:58:54.900076 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:54.900039    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6944578694-ss64h"]
Jul 03 09:58:54.900315 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:54.900263    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6944578694-ss64h" podUID="ad173716-8a45-4444-af9d-1258893c77b5" containerName="authorino" containerID="cri-o://949e2028ae9588480c3426ee0737e97f8d5b6fc6b4b980dad5e5632402140ef2" gracePeriod=30
Jul 03 09:58:55.169469 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.169441    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6944578694-ss64h"
Jul 03 09:58:55.315097 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.315048    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad173716-8a45-4444-af9d-1258893c77b5-tls-cert\") pod \"ad173716-8a45-4444-af9d-1258893c77b5\" (UID: \"ad173716-8a45-4444-af9d-1258893c77b5\") "
Jul 03 09:58:55.315097 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.315099    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f4s9\" (UniqueName: \"kubernetes.io/projected/ad173716-8a45-4444-af9d-1258893c77b5-kube-api-access-6f4s9\") pod \"ad173716-8a45-4444-af9d-1258893c77b5\" (UID: \"ad173716-8a45-4444-af9d-1258893c77b5\") "
Jul 03 09:58:55.317321 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.317286    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad173716-8a45-4444-af9d-1258893c77b5-kube-api-access-6f4s9" (OuterVolumeSpecName: "kube-api-access-6f4s9") pod "ad173716-8a45-4444-af9d-1258893c77b5" (UID: "ad173716-8a45-4444-af9d-1258893c77b5"). InnerVolumeSpecName "kube-api-access-6f4s9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 09:58:55.326167 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.326138    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad173716-8a45-4444-af9d-1258893c77b5-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ad173716-8a45-4444-af9d-1258893c77b5" (UID: "ad173716-8a45-4444-af9d-1258893c77b5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 09:58:55.416235 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.416141    2571 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad173716-8a45-4444-af9d-1258893c77b5-tls-cert\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:58:55.416235 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.416181    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f4s9\" (UniqueName: \"kubernetes.io/projected/ad173716-8a45-4444-af9d-1258893c77b5-kube-api-access-6f4s9\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 09:58:55.840112 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.840020    2571 generic.go:356] "Generic (PLEG): container finished" podID="ad173716-8a45-4444-af9d-1258893c77b5" containerID="949e2028ae9588480c3426ee0737e97f8d5b6fc6b4b980dad5e5632402140ef2" exitCode=0
Jul 03 09:58:55.840112 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.840056    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6944578694-ss64h"
Jul 03 09:58:55.840694 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.840103    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6944578694-ss64h" event={"ID":"ad173716-8a45-4444-af9d-1258893c77b5","Type":"ContainerDied","Data":"949e2028ae9588480c3426ee0737e97f8d5b6fc6b4b980dad5e5632402140ef2"}
Jul 03 09:58:55.840694 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.840154    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6944578694-ss64h" event={"ID":"ad173716-8a45-4444-af9d-1258893c77b5","Type":"ContainerDied","Data":"068e5fc9f49b21ee993f18c9107293f77e53216815a00741ab8ac0e79002ecec"}
Jul 03 09:58:55.840694 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.840177    2571 scope.go:117] "RemoveContainer" containerID="949e2028ae9588480c3426ee0737e97f8d5b6fc6b4b980dad5e5632402140ef2"
Jul 03 09:58:55.850284 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.850264    2571 scope.go:117] "RemoveContainer" containerID="949e2028ae9588480c3426ee0737e97f8d5b6fc6b4b980dad5e5632402140ef2"
Jul 03 09:58:55.850584 ip-10-0-138-129 kubenswrapper[2571]: E0703 09:58:55.850566    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"949e2028ae9588480c3426ee0737e97f8d5b6fc6b4b980dad5e5632402140ef2\": container with ID starting with 949e2028ae9588480c3426ee0737e97f8d5b6fc6b4b980dad5e5632402140ef2 not found: ID does not exist" containerID="949e2028ae9588480c3426ee0737e97f8d5b6fc6b4b980dad5e5632402140ef2"
Jul 03 09:58:55.850630 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.850593    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"949e2028ae9588480c3426ee0737e97f8d5b6fc6b4b980dad5e5632402140ef2"} err="failed to get container status \"949e2028ae9588480c3426ee0737e97f8d5b6fc6b4b980dad5e5632402140ef2\": rpc error: code = NotFound desc = could not find container \"949e2028ae9588480c3426ee0737e97f8d5b6fc6b4b980dad5e5632402140ef2\": container with ID starting with 949e2028ae9588480c3426ee0737e97f8d5b6fc6b4b980dad5e5632402140ef2 not found: ID does not exist"
Jul 03 09:58:55.863646 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.863618    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6944578694-ss64h"]
Jul 03 09:58:55.876720 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:55.876692    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6944578694-ss64h"]
Jul 03 09:58:56.764893 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:58:56.764827    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad173716-8a45-4444-af9d-1258893c77b5" path="/var/lib/kubelet/pods/ad173716-8a45-4444-af9d-1258893c77b5/volumes"
Jul 03 09:59:23.917242 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:59:23.917204    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:59:33.111139 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:59:33.111101    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:59:40.698872 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:59:40.698744    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 09:59:45.401187 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:59:45.401143    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 09:59:53.728378 ip-10-0-138-129 kubenswrapper[2571]: I0703 09:59:53.728339    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:00:00.149546 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:00.149507    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29717880-j5nbw"]
Jul 03 10:00:00.149972 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:00.149955    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad173716-8a45-4444-af9d-1258893c77b5" containerName="authorino"
Jul 03 10:00:00.149972 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:00.149968    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad173716-8a45-4444-af9d-1258893c77b5" containerName="authorino"
Jul 03 10:00:00.150068 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:00.150057    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad173716-8a45-4444-af9d-1258893c77b5" containerName="authorino"
Jul 03 10:00:00.152749 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:00.152734    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717880-j5nbw"
Jul 03 10:00:00.155270 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:00.155249    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-bh87n\""
Jul 03 10:00:00.162651 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:00.162626    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717880-j5nbw"]
Jul 03 10:00:00.286059 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:00.286026    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc465\" (UniqueName: \"kubernetes.io/projected/7c319aa1-3f7f-40a5-a4a3-f938749d4103-kube-api-access-hc465\") pod \"maas-api-key-cleanup-29717880-j5nbw\" (UID: \"7c319aa1-3f7f-40a5-a4a3-f938749d4103\") " pod="opendatahub/maas-api-key-cleanup-29717880-j5nbw"
Jul 03 10:00:00.387265 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:00.387228    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc465\" (UniqueName: \"kubernetes.io/projected/7c319aa1-3f7f-40a5-a4a3-f938749d4103-kube-api-access-hc465\") pod \"maas-api-key-cleanup-29717880-j5nbw\" (UID: \"7c319aa1-3f7f-40a5-a4a3-f938749d4103\") " pod="opendatahub/maas-api-key-cleanup-29717880-j5nbw"
Jul 03 10:00:00.397884 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:00.397861    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc465\" (UniqueName: \"kubernetes.io/projected/7c319aa1-3f7f-40a5-a4a3-f938749d4103-kube-api-access-hc465\") pod \"maas-api-key-cleanup-29717880-j5nbw\" (UID: \"7c319aa1-3f7f-40a5-a4a3-f938749d4103\") " pod="opendatahub/maas-api-key-cleanup-29717880-j5nbw"
Jul 03 10:00:00.465404 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:00.465320    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717880-j5nbw"
Jul 03 10:00:00.602121 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:00.602096    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717880-j5nbw"]
Jul 03 10:00:00.604612 ip-10-0-138-129 kubenswrapper[2571]: W0703 10:00:00.604584    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c319aa1_3f7f_40a5_a4a3_f938749d4103.slice/crio-986a0c29574f1ae85ab86a2b49d56159ff3e62f286490c511c77e7309c8037e7 WatchSource:0}: Error finding container 986a0c29574f1ae85ab86a2b49d56159ff3e62f286490c511c77e7309c8037e7: Status 404 returned error can't find the container with id 986a0c29574f1ae85ab86a2b49d56159ff3e62f286490c511c77e7309c8037e7
Jul 03 10:00:01.183824 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:01.183784    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717880-j5nbw" event={"ID":"7c319aa1-3f7f-40a5-a4a3-f938749d4103","Type":"ContainerStarted","Data":"986a0c29574f1ae85ab86a2b49d56159ff3e62f286490c511c77e7309c8037e7"}
Jul 03 10:00:05.202554 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:05.202522    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:00:07.056620 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:07.056548    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:00:09.370749 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:09.370713    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:00:12.241733 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:12.241687    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717880-j5nbw" event={"ID":"7c319aa1-3f7f-40a5-a4a3-f938749d4103","Type":"ContainerStarted","Data":"c92eb2985298e8447393b944e8c9bf1b5650f306bffdf621b4f4d39fe06fa3c4"}
Jul 03 10:00:12.261413 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:12.261365    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29717880-j5nbw" podStartSLOduration=1.049444803 podStartE2EDuration="12.261348024s" totalImagesPullingTime="11.211903221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:00:12.259658063 +0000 UTC m=+1014.124946784" watchObservedRunningTime="2026-07-03 10:00:12.261348024 +0000 UTC m=+1014.126636679"
Jul 03 10:00:13.248522 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:13.248488    2571 generic.go:356] "Generic (PLEG): container finished" podID="7c319aa1-3f7f-40a5-a4a3-f938749d4103" containerID="c92eb2985298e8447393b944e8c9bf1b5650f306bffdf621b4f4d39fe06fa3c4" exitCode=0
Jul 03 10:00:13.248940 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:13.248555    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717880-j5nbw" event={"ID":"7c319aa1-3f7f-40a5-a4a3-f938749d4103","Type":"ContainerDied","Data":"c92eb2985298e8447393b944e8c9bf1b5650f306bffdf621b4f4d39fe06fa3c4"}
Jul 03 10:00:14.389968 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:14.389940    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717880-j5nbw"
Jul 03 10:00:14.405071 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:14.405045    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc465\" (UniqueName: \"kubernetes.io/projected/7c319aa1-3f7f-40a5-a4a3-f938749d4103-kube-api-access-hc465\") pod \"7c319aa1-3f7f-40a5-a4a3-f938749d4103\" (UID: \"7c319aa1-3f7f-40a5-a4a3-f938749d4103\") "
Jul 03 10:00:14.407374 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:14.407323    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c319aa1-3f7f-40a5-a4a3-f938749d4103-kube-api-access-hc465" (OuterVolumeSpecName: "kube-api-access-hc465") pod "7c319aa1-3f7f-40a5-a4a3-f938749d4103" (UID: "7c319aa1-3f7f-40a5-a4a3-f938749d4103"). InnerVolumeSpecName "kube-api-access-hc465". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 10:00:14.506552 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:14.506463    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc465\" (UniqueName: \"kubernetes.io/projected/7c319aa1-3f7f-40a5-a4a3-f938749d4103-kube-api-access-hc465\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 10:00:15.260072 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:15.260034    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717880-j5nbw" event={"ID":"7c319aa1-3f7f-40a5-a4a3-f938749d4103","Type":"ContainerDied","Data":"986a0c29574f1ae85ab86a2b49d56159ff3e62f286490c511c77e7309c8037e7"}
Jul 03 10:00:15.260072 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:15.260059    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717880-j5nbw"
Jul 03 10:00:15.260280 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:15.260088    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="986a0c29574f1ae85ab86a2b49d56159ff3e62f286490c511c77e7309c8037e7"
Jul 03 10:00:32.810382 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:32.810348    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:00:42.006882 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:42.006828    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:00:46.167480 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:46.167423    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:00:46.529895 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:46.529741    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:00:46.599678 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:46.599642    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:00:52.107605 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:52.107558    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:00:52.223148 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:52.223116    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:00:52.463153 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:52.463092    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:00:52.560651 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:52.560612    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:00:58.215397 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:58.215345    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:00:58.322618 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:00:58.322567    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:01:15.695576 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:01:15.695491    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:01:18.784425 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:01:18.784395    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:01:24.607488 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:01:24.607454    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:01:33.914047 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:01:33.914009    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:01:45.008933 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:01:45.008895    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:01:54.159906 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:01:54.159869    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:02:05.601983 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:02:05.601945    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:02:15.503282 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:02:15.503244    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:02:24.905055 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:02:24.905018    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:02:47.115276 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:02:47.115233    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:02:50.852102 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:02:50.852064    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:02:57.396114 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:02:57.396076    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:03:18.690473 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:03:18.690443    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 10:03:18.695200 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:03:18.695179    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 10:03:20.901067 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:03:20.901028    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:03:31.702709 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:03:31.702669    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:03:46.001467 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:03:46.001427    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:03:57.012853 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:03:57.012794    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:04:06.306008 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:06.305970    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:04:15.799055 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:15.798974    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:04:24.209284 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:24.209243    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:04:32.953560 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:32.953526    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:04:51.218464 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:51.218424    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"]
Jul 03 10:04:51.218909 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:51.218638    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2" podUID="e62bbfb4-22a3-47eb-b82b-4a646b93ce57" containerName="manager" containerID="cri-o://58efb3e8debdbe5e4ad280d7b62b77c26bb9b5006819f8ee0b37c8604ca28d62" gracePeriod=10
Jul 03 10:04:54.717462 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:54.717427    2571 generic.go:356] "Generic (PLEG): container finished" podID="e62bbfb4-22a3-47eb-b82b-4a646b93ce57" containerID="58efb3e8debdbe5e4ad280d7b62b77c26bb9b5006819f8ee0b37c8604ca28d62" exitCode=0
Jul 03 10:04:54.717808 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:54.717496    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2" event={"ID":"e62bbfb4-22a3-47eb-b82b-4a646b93ce57","Type":"ContainerDied","Data":"58efb3e8debdbe5e4ad280d7b62b77c26bb9b5006819f8ee0b37c8604ca28d62"}
Jul 03 10:04:54.776564 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:54.776539    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"
Jul 03 10:04:54.945081 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:54.945049    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e62bbfb4-22a3-47eb-b82b-4a646b93ce57-extensions-socket-volume\") pod \"e62bbfb4-22a3-47eb-b82b-4a646b93ce57\" (UID: \"e62bbfb4-22a3-47eb-b82b-4a646b93ce57\") "
Jul 03 10:04:54.945273 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:54.945092    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwppr\" (UniqueName: \"kubernetes.io/projected/e62bbfb4-22a3-47eb-b82b-4a646b93ce57-kube-api-access-bwppr\") pod \"e62bbfb4-22a3-47eb-b82b-4a646b93ce57\" (UID: \"e62bbfb4-22a3-47eb-b82b-4a646b93ce57\") "
Jul 03 10:04:54.945458 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:54.945425    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e62bbfb4-22a3-47eb-b82b-4a646b93ce57-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e62bbfb4-22a3-47eb-b82b-4a646b93ce57" (UID: "e62bbfb4-22a3-47eb-b82b-4a646b93ce57"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 10:04:54.947305 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:54.947271    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e62bbfb4-22a3-47eb-b82b-4a646b93ce57-kube-api-access-bwppr" (OuterVolumeSpecName: "kube-api-access-bwppr") pod "e62bbfb4-22a3-47eb-b82b-4a646b93ce57" (UID: "e62bbfb4-22a3-47eb-b82b-4a646b93ce57"). InnerVolumeSpecName "kube-api-access-bwppr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 10:04:55.046072 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:55.046035    2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e62bbfb4-22a3-47eb-b82b-4a646b93ce57-extensions-socket-volume\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 10:04:55.046072 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:55.046069    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwppr\" (UniqueName: \"kubernetes.io/projected/e62bbfb4-22a3-47eb-b82b-4a646b93ce57-kube-api-access-bwppr\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 10:04:55.724882 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:55.724826    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"
Jul 03 10:04:55.724882 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:55.724861    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2" event={"ID":"e62bbfb4-22a3-47eb-b82b-4a646b93ce57","Type":"ContainerDied","Data":"2fbf5e1ab53cddbb9bb0ee10950c62b4bfa0e6ead33ffc55fea34d17339b8e7d"}
Jul 03 10:04:55.725454 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:55.724922    2571 scope.go:117] "RemoveContainer" containerID="58efb3e8debdbe5e4ad280d7b62b77c26bb9b5006819f8ee0b37c8604ca28d62"
Jul 03 10:04:55.747994 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:55.747966    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"]
Jul 03 10:04:55.752224 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:55.752204    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bvwq2"]
Jul 03 10:04:56.765293 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:04:56.765259    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e62bbfb4-22a3-47eb-b82b-4a646b93ce57" path="/var/lib/kubelet/pods/e62bbfb4-22a3-47eb-b82b-4a646b93ce57/volumes"
Jul 03 10:05:52.350228 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.350188    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j"]
Jul 03 10:05:52.350739 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.350591    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e62bbfb4-22a3-47eb-b82b-4a646b93ce57" containerName="manager"
Jul 03 10:05:52.350739 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.350603    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e62bbfb4-22a3-47eb-b82b-4a646b93ce57" containerName="manager"
Jul 03 10:05:52.350739 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.350621    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c319aa1-3f7f-40a5-a4a3-f938749d4103" containerName="cleanup"
Jul 03 10:05:52.350739 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.350626    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c319aa1-3f7f-40a5-a4a3-f938749d4103" containerName="cleanup"
Jul 03 10:05:52.350739 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.350702    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e62bbfb4-22a3-47eb-b82b-4a646b93ce57" containerName="manager"
Jul 03 10:05:52.350739 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.350712    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c319aa1-3f7f-40a5-a4a3-f938749d4103" containerName="cleanup"
Jul 03 10:05:52.353636 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.353619    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j"
Jul 03 10:05:52.356588 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.356566    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-blwqx\""
Jul 03 10:05:52.372243 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.372219    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j"]
Jul 03 10:05:52.438638 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.438605    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4b86c35-c428-4375-901f-2c470e7621a8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wth9j\" (UID: \"e4b86c35-c428-4375-901f-2c470e7621a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j"
Jul 03 10:05:52.438790 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.438699    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7xm5\" (UniqueName: \"kubernetes.io/projected/e4b86c35-c428-4375-901f-2c470e7621a8-kube-api-access-n7xm5\") pod \"kuadrant-operator-controller-manager-86878c769f-wth9j\" (UID: \"e4b86c35-c428-4375-901f-2c470e7621a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j"
Jul 03 10:05:52.539333 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.539289    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7xm5\" (UniqueName: \"kubernetes.io/projected/e4b86c35-c428-4375-901f-2c470e7621a8-kube-api-access-n7xm5\") pod \"kuadrant-operator-controller-manager-86878c769f-wth9j\" (UID: \"e4b86c35-c428-4375-901f-2c470e7621a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j"
Jul 03 10:05:52.539530 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.539389    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4b86c35-c428-4375-901f-2c470e7621a8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wth9j\" (UID: \"e4b86c35-c428-4375-901f-2c470e7621a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j"
Jul 03 10:05:52.539779 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.539759    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4b86c35-c428-4375-901f-2c470e7621a8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wth9j\" (UID: \"e4b86c35-c428-4375-901f-2c470e7621a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j"
Jul 03 10:05:52.550848 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.550819    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7xm5\" (UniqueName: \"kubernetes.io/projected/e4b86c35-c428-4375-901f-2c470e7621a8-kube-api-access-n7xm5\") pod \"kuadrant-operator-controller-manager-86878c769f-wth9j\" (UID: \"e4b86c35-c428-4375-901f-2c470e7621a8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j"
Jul 03 10:05:52.664260 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.664223    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j"
Jul 03 10:05:52.815591 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:52.815566    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j"]
Jul 03 10:05:52.817378 ip-10-0-138-129 kubenswrapper[2571]: W0703 10:05:52.817351    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4b86c35_c428_4375_901f_2c470e7621a8.slice/crio-41d6a8416578cb517896460e71b3625a87fe69fed935daa98aaae74dbe133c89 WatchSource:0}: Error finding container 41d6a8416578cb517896460e71b3625a87fe69fed935daa98aaae74dbe133c89: Status 404 returned error can't find the container with id 41d6a8416578cb517896460e71b3625a87fe69fed935daa98aaae74dbe133c89
Jul 03 10:05:53.004310 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:53.004214    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j" event={"ID":"e4b86c35-c428-4375-901f-2c470e7621a8","Type":"ContainerStarted","Data":"a79bc2759b70cdb999a697e860770ae5d0adedad3b42fdff539c4ca48a88c4ce"}
Jul 03 10:05:53.004310 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:53.004270    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j" event={"ID":"e4b86c35-c428-4375-901f-2c470e7621a8","Type":"ContainerStarted","Data":"41d6a8416578cb517896460e71b3625a87fe69fed935daa98aaae74dbe133c89"}
Jul 03 10:05:53.004527 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:53.004337    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j"
Jul 03 10:05:53.027129 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:05:53.027075    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j" podStartSLOduration=1.027060445 podStartE2EDuration="1.027060445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:05:53.026080056 +0000 UTC m=+1354.891368714" watchObservedRunningTime="2026-07-03 10:05:53.027060445 +0000 UTC m=+1354.892349100"
Jul 03 10:06:04.012457 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:06:04.012418    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wth9j"
Jul 03 10:06:12.463016 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:06:12.462977    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:06:12.500160 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:06:12.500113    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:06:22.906687 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:06:22.906649    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:06:33.608008 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:06:33.607970    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:06:44.106723 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:06:44.106688    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:06:55.612331 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:06:55.612291    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:07:05.454743 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:07:05.454704    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:07:15.308120 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:07:15.308033    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:07:25.406529 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:07:25.406490    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:07:36.810239 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:07:36.810204    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:07:44.113405 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:07:44.113367    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:07:52.808591 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:07:52.808556    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:08:02.804522 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:08:02.804485    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:08:18.721439 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:08:18.721397    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 10:08:18.727135 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:08:18.727114    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 10:08:20.009608 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:08:20.009573    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:08:29.203119 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:08:29.203082    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:08:33.166434 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:08:33.166385    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:08:47.119996 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:08:47.119954    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:09:04.598333 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:09:04.598294    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:09:16.095287 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:09:16.095249    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:09:25.022257 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:09:25.022218    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:09:35.004928 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:09:35.004890    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:09:44.099552 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:09:44.099512    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:09:53.511121 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:09:53.511074    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:10:06.600975 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:10:06.600920    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:10:15.307716 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:10:15.307631    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:10:24.715467 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:10:24.715429    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:10:36.196184 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:10:36.196147    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:10:48.734665 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:10:48.734621    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:10:57.444275 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:10:57.444234    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:11:09.116690 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:11:09.116659    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:11:20.816253 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:11:20.816221    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:11:33.418921 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:11:33.418886    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:11:36.434912 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:11:36.434879    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:11:39.226807 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:11:39.226774    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:11:42.109348 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:11:42.109265    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:11:46.643089 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:11:46.643050    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:11:46.944224 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:11:46.944195    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:11:51.766066 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:11:51.766029    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:11:54.886677 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:11:54.886635    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:11:55.311069 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:11:55.311034    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:12:00.011891 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:00.011859    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:02.876881 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:02.876815    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:02.976271 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:02.976234    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:03.357049 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:03.357016    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:03.752677 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:03.752639    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:12:09.383640 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:09.383607    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:09.487878 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:09.487823    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:09.688895 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:09.688850    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:09.787948 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:09.787911    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:10.854292 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:10.854255    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:10.956098 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:10.956051    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:12.626429 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:12.626389    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:12.727327 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:12.727292    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:13.341068 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:13.341036    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:13.508505 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:13.508475    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:12:15.916429 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:15.916402    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:16.099005 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:16.098970    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:16.185129 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:16.185038    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:16.341708 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:16.341665    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:20.121414 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:20.121365    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:22.006372 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:22.006339    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:12:22.317129 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:22.317046    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:22.420878 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:22.420822    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:22.550888 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:22.550854    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:22.622736 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:22.622710    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:23.822305 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:23.822276    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:23.924135 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:23.924104    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:24.032761 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:24.032724    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:24.269074 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:24.269042    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:24.684040 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:24.683991    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:24.783857 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:24.783785    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:25.759955 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:25.759897    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:28.838682 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:28.838643    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:29.268397 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:29.268342    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:35.116443 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:35.116393    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:35.217583 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:35.217547    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:37.148886 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:37.148809    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:41.545715 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:41.545675    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:41.643850 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:41.643802    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:41.774501 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:41.774470    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:41.885084 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:41.884995    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:43.072547 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.072505    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:43.171287 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.171226    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:43.215487 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.215459    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"]
Jul 03 10:12:43.219130 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.219112    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:43.223699 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.223681    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-1b6948bd-serving-cert\""
Jul 03 10:12:43.224411 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.224397    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-bh87n\""
Jul 03 10:12:43.242527 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.242501    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"]
Jul 03 10:12:43.273673 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.273644    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:43.332771 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.332691    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3344e406-933e-4613-a745-3997ebe257b5-maas-api-tls\") pod \"maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr\" (UID: \"3344e406-933e-4613-a745-3997ebe257b5\") " pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:43.332771 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.332743    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3344e406-933e-4613-a745-3997ebe257b5-openshift-service-ca\") pod \"maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr\" (UID: \"3344e406-933e-4613-a745-3997ebe257b5\") " pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:43.333020 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.332867    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnc8r\" (UniqueName: \"kubernetes.io/projected/3344e406-933e-4613-a745-3997ebe257b5-kube-api-access-wnc8r\") pod \"maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr\" (UID: \"3344e406-933e-4613-a745-3997ebe257b5\") " pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:43.433989 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.433958    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3344e406-933e-4613-a745-3997ebe257b5-maas-api-tls\") pod \"maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr\" (UID: \"3344e406-933e-4613-a745-3997ebe257b5\") " pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:43.434175 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.434011    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3344e406-933e-4613-a745-3997ebe257b5-openshift-service-ca\") pod \"maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr\" (UID: \"3344e406-933e-4613-a745-3997ebe257b5\") " pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:43.434175 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.434063    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnc8r\" (UniqueName: \"kubernetes.io/projected/3344e406-933e-4613-a745-3997ebe257b5-kube-api-access-wnc8r\") pod \"maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr\" (UID: \"3344e406-933e-4613-a745-3997ebe257b5\") " pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:43.434175 ip-10-0-138-129 kubenswrapper[2571]: E0703 10:12:43.434097    2571 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-1b6948bd-serving-cert: secret "maas-api-e2e-derive-1b6948bd-serving-cert" not found
Jul 03 10:12:43.434175 ip-10-0-138-129 kubenswrapper[2571]: E0703 10:12:43.434167    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3344e406-933e-4613-a745-3997ebe257b5-maas-api-tls podName:3344e406-933e-4613-a745-3997ebe257b5 nodeName:}" failed. No retries permitted until 2026-07-03 10:12:43.934150739 +0000 UTC m=+1765.799439372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3344e406-933e-4613-a745-3997ebe257b5-maas-api-tls") pod "maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr" (UID: "3344e406-933e-4613-a745-3997ebe257b5") : secret "maas-api-e2e-derive-1b6948bd-serving-cert" not found
Jul 03 10:12:43.434660 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.434640    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3344e406-933e-4613-a745-3997ebe257b5-openshift-service-ca\") pod \"maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr\" (UID: \"3344e406-933e-4613-a745-3997ebe257b5\") " pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:43.443031 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.442998    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnc8r\" (UniqueName: \"kubernetes.io/projected/3344e406-933e-4613-a745-3997ebe257b5-kube-api-access-wnc8r\") pod \"maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr\" (UID: \"3344e406-933e-4613-a745-3997ebe257b5\") " pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:43.568617 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.568585    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:43.713950 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.713920    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:43.811195 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.811137    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:43.940220 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.940179    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3344e406-933e-4613-a745-3997ebe257b5-maas-api-tls\") pod \"maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr\" (UID: \"3344e406-933e-4613-a745-3997ebe257b5\") " pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:43.942742 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:43.942707    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3344e406-933e-4613-a745-3997ebe257b5-maas-api-tls\") pod \"maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr\" (UID: \"3344e406-933e-4613-a745-3997ebe257b5\") " pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:44.130320 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:44.130225    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:44.286269 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:44.286241    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"]
Jul 03 10:12:44.287342 ip-10-0-138-129 kubenswrapper[2571]: W0703 10:12:44.287313    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3344e406_933e_4613_a745_3997ebe257b5.slice/crio-f886395f4dae779349e28a49931e419b3910115493c3f5462f42e7859ba267dc WatchSource:0}: Error finding container f886395f4dae779349e28a49931e419b3910115493c3f5462f42e7859ba267dc: Status 404 returned error can't find the container with id f886395f4dae779349e28a49931e419b3910115493c3f5462f42e7859ba267dc
Jul 03 10:12:44.288678 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:44.288659    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 10:12:44.794498 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:44.794469    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:45.161408 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:45.161363    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr" event={"ID":"3344e406-933e-4613-a745-3997ebe257b5","Type":"ContainerStarted","Data":"f886395f4dae779349e28a49931e419b3910115493c3f5462f42e7859ba267dc"}
Jul 03 10:12:45.839808 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:45.839784    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:12:46.168133 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:46.168102    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr" event={"ID":"3344e406-933e-4613-a745-3997ebe257b5","Type":"ContainerStarted","Data":"af8a27e6039f7a05950781e6c1ec77ebe6da48d2c8fc553349b747fc117e0ce6"}
Jul 03 10:12:46.168603 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:46.168177    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:46.189024 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:46.188971    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr" podStartSLOduration=1.692244037 podStartE2EDuration="3.188955375s" totalImagesPullingTime="1.496711338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:12:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:12:46.188070769 +0000 UTC m=+1768.053359425" watchObservedRunningTime="2026-07-03 10:12:46.188955375 +0000 UTC m=+1768.054244061"
Jul 03 10:12:47.427279 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:47.427240    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:47.610941 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:47.610890    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:47.711172 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:47.711096    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:47.789643 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:47.789613    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"]
Jul 03 10:12:47.879668 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:47.879638    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:48.178812 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:48.178769    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr" podUID="3344e406-933e-4613-a745-3997ebe257b5" containerName="maas-api" containerID="cri-o://af8a27e6039f7a05950781e6c1ec77ebe6da48d2c8fc553349b747fc117e0ce6" gracePeriod=30
Jul 03 10:12:48.419015 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:48.418992    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:48.485271 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:48.485193    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3344e406-933e-4613-a745-3997ebe257b5-maas-api-tls\") pod \"3344e406-933e-4613-a745-3997ebe257b5\" (UID: \"3344e406-933e-4613-a745-3997ebe257b5\") "
Jul 03 10:12:48.485655 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:48.485289    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3344e406-933e-4613-a745-3997ebe257b5-openshift-service-ca\") pod \"3344e406-933e-4613-a745-3997ebe257b5\" (UID: \"3344e406-933e-4613-a745-3997ebe257b5\") "
Jul 03 10:12:48.485655 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:48.485307    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnc8r\" (UniqueName: \"kubernetes.io/projected/3344e406-933e-4613-a745-3997ebe257b5-kube-api-access-wnc8r\") pod \"3344e406-933e-4613-a745-3997ebe257b5\" (UID: \"3344e406-933e-4613-a745-3997ebe257b5\") "
Jul 03 10:12:48.485655 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:48.485609    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3344e406-933e-4613-a745-3997ebe257b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3344e406-933e-4613-a745-3997ebe257b5" (UID: "3344e406-933e-4613-a745-3997ebe257b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 10:12:48.487603 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:48.487566    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3344e406-933e-4613-a745-3997ebe257b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3344e406-933e-4613-a745-3997ebe257b5" (UID: "3344e406-933e-4613-a745-3997ebe257b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 10:12:48.487721 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:48.487697    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3344e406-933e-4613-a745-3997ebe257b5-kube-api-access-wnc8r" (OuterVolumeSpecName: "kube-api-access-wnc8r") pod "3344e406-933e-4613-a745-3997ebe257b5" (UID: "3344e406-933e-4613-a745-3997ebe257b5"). InnerVolumeSpecName "kube-api-access-wnc8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 10:12:48.586124 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:48.586092    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3344e406-933e-4613-a745-3997ebe257b5-maas-api-tls\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 10:12:48.586124 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:48.586120    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3344e406-933e-4613-a745-3997ebe257b5-openshift-service-ca\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 10:12:48.586124 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:48.586133    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnc8r\" (UniqueName: \"kubernetes.io/projected/3344e406-933e-4613-a745-3997ebe257b5-kube-api-access-wnc8r\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 10:12:48.988034 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:48.988004    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:49.183823 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:49.183785    2571 generic.go:356] "Generic (PLEG): container finished" podID="3344e406-933e-4613-a745-3997ebe257b5" containerID="af8a27e6039f7a05950781e6c1ec77ebe6da48d2c8fc553349b747fc117e0ce6" exitCode=0
Jul 03 10:12:49.184000 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:49.183846    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"
Jul 03 10:12:49.184000 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:49.183876    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr" event={"ID":"3344e406-933e-4613-a745-3997ebe257b5","Type":"ContainerDied","Data":"af8a27e6039f7a05950781e6c1ec77ebe6da48d2c8fc553349b747fc117e0ce6"}
Jul 03 10:12:49.184000 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:49.183930    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr" event={"ID":"3344e406-933e-4613-a745-3997ebe257b5","Type":"ContainerDied","Data":"f886395f4dae779349e28a49931e419b3910115493c3f5462f42e7859ba267dc"}
Jul 03 10:12:49.184000 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:49.183952    2571 scope.go:117] "RemoveContainer" containerID="af8a27e6039f7a05950781e6c1ec77ebe6da48d2c8fc553349b747fc117e0ce6"
Jul 03 10:12:49.192700 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:49.192691    2571 scope.go:117] "RemoveContainer" containerID="af8a27e6039f7a05950781e6c1ec77ebe6da48d2c8fc553349b747fc117e0ce6"
Jul 03 10:12:49.192944 ip-10-0-138-129 kubenswrapper[2571]: E0703 10:12:49.192924    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af8a27e6039f7a05950781e6c1ec77ebe6da48d2c8fc553349b747fc117e0ce6\": container with ID starting with af8a27e6039f7a05950781e6c1ec77ebe6da48d2c8fc553349b747fc117e0ce6 not found: ID does not exist" containerID="af8a27e6039f7a05950781e6c1ec77ebe6da48d2c8fc553349b747fc117e0ce6"
Jul 03 10:12:49.192995 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:49.192950    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af8a27e6039f7a05950781e6c1ec77ebe6da48d2c8fc553349b747fc117e0ce6"} err="failed to get container status \"af8a27e6039f7a05950781e6c1ec77ebe6da48d2c8fc553349b747fc117e0ce6\": rpc error: code = NotFound desc = could not find container \"af8a27e6039f7a05950781e6c1ec77ebe6da48d2c8fc553349b747fc117e0ce6\": container with ID starting with af8a27e6039f7a05950781e6c1ec77ebe6da48d2c8fc553349b747fc117e0ce6 not found: ID does not exist"
Jul 03 10:12:49.202354 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:49.202333    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"]
Jul 03 10:12:49.207230 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:49.207212    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-1b6948bd-7798bc569f-k65jr"]
Jul 03 10:12:50.764823 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:50.764790    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3344e406-933e-4613-a745-3997ebe257b5" path="/var/lib/kubelet/pods/3344e406-933e-4613-a745-3997ebe257b5/volumes"
Jul 03 10:12:54.093985 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:54.093955    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:54.114037 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:54.114006    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:12:54.190666 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:54.190599    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:54.576644 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:54.576604    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:54.676800 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:54.676772    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:54.776271 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:54.776228    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:56.125319 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:56.125286    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:12:58.059079 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:12:58.059022    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:02.167031 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:02.166992    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:02.272493 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:02.272449    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:02.392174 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:02.392136    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:02.491644 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:02.491511    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:02.963444 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:02.963392    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:18.750915 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:18.750885    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 10:13:18.757787 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:18.757763    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 10:13:28.886081 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:28.886035    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:34.783243 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:34.783206    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:34.887347 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:34.887314    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:34.980355 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:34.980322    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:50.882721 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:50.882689    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:57.099001 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:57.098968    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:57.169019 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:57.168996    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:57.267884 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:57.267855    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:13:57.369133 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:13:57.369038    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:07.860315 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:07.860283    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:08.624272 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:08.624241    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:14.735014 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:14.734982    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:14.807447 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:14.807417    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:14.915608 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:14.915579    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:15.034227 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:15.034142    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:15.826270 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:15.826238    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:15.923676 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:15.923642    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:16.028273 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:16.028238    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:17.657271 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:17.657237    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:18.326276 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:18.326243    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:14:23.783104 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:23.783066    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:23.886651 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:23.886616    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:24.476150 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:24.476118    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:26.913223 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:26.913190    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:14:30.458159 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:30.458128    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:30.535453 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:30.535418    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:30.699644 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:30.699612    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:31.069192 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:31.069155    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:31.570433 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:31.570395    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:33.986723 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:33.986685    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:34.129289 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:34.129248    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:34.212979 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:34.212946    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:34.411327 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:34.411296    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:34.921502 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:34.921470    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:35.027617 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:35.027583    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:38.339359 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:38.339326    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:39.021267 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:39.021239    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:39.118823 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:39.118794    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:39.430000 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:39.429969    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:39.914489 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:39.914459    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:14:45.476513 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:45.476483    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:45.578493 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:45.578460    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:45.904426 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:45.904392    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:46.271413 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:46.271334    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:46.371167 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:46.371137    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:50.602467 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:50.602424    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:14:52.153518 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:52.153481    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:52.276434 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:52.276401    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:52.504119 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:52.504037    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:52.668565 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:52.668532    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:52.799691 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:52.799607    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:52.883894 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:52.883859    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:53.005417 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:53.005377    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:58.865902 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:58.865872    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:58.989387 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:58.989348    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:59.072401 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:59.072365    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:14:59.214724 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:14:59.214693    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:00.129324 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.129292    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29717895-x5fdk"]
Jul 03 10:15:00.129713 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.129700    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3344e406-933e-4613-a745-3997ebe257b5" containerName="maas-api"
Jul 03 10:15:00.129713 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.129714    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3344e406-933e-4613-a745-3997ebe257b5" containerName="maas-api"
Jul 03 10:15:00.129789 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.129783    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3344e406-933e-4613-a745-3997ebe257b5" containerName="maas-api"
Jul 03 10:15:00.132991 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.132974    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717895-x5fdk"
Jul 03 10:15:00.135555 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.135531    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-bh87n\""
Jul 03 10:15:00.138287 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.138261    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717895-x5fdk"]
Jul 03 10:15:00.244413 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.244379    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw9t6\" (UniqueName: \"kubernetes.io/projected/e5e3957c-e27b-4bb1-9ab0-cd7ecd612144-kube-api-access-gw9t6\") pod \"maas-api-key-cleanup-29717895-x5fdk\" (UID: \"e5e3957c-e27b-4bb1-9ab0-cd7ecd612144\") " pod="opendatahub/maas-api-key-cleanup-29717895-x5fdk"
Jul 03 10:15:00.345322 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.345291    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw9t6\" (UniqueName: \"kubernetes.io/projected/e5e3957c-e27b-4bb1-9ab0-cd7ecd612144-kube-api-access-gw9t6\") pod \"maas-api-key-cleanup-29717895-x5fdk\" (UID: \"e5e3957c-e27b-4bb1-9ab0-cd7ecd612144\") " pod="opendatahub/maas-api-key-cleanup-29717895-x5fdk"
Jul 03 10:15:00.359528 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.359499    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw9t6\" (UniqueName: \"kubernetes.io/projected/e5e3957c-e27b-4bb1-9ab0-cd7ecd612144-kube-api-access-gw9t6\") pod \"maas-api-key-cleanup-29717895-x5fdk\" (UID: \"e5e3957c-e27b-4bb1-9ab0-cd7ecd612144\") " pod="opendatahub/maas-api-key-cleanup-29717895-x5fdk"
Jul 03 10:15:00.445829 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.445801    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717895-x5fdk"
Jul 03 10:15:00.580878 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.580851    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717895-x5fdk"]
Jul 03 10:15:00.582994 ip-10-0-138-129 kubenswrapper[2571]: W0703 10:15:00.582969    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e3957c_e27b_4bb1_9ab0_cd7ecd612144.slice/crio-ef72911581c4e146c1b911bd32024c44f3418a756070a753c1859352fa58e82f WatchSource:0}: Error finding container ef72911581c4e146c1b911bd32024c44f3418a756070a753c1859352fa58e82f: Status 404 returned error can't find the container with id ef72911581c4e146c1b911bd32024c44f3418a756070a753c1859352fa58e82f
Jul 03 10:15:00.865384 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.865357    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717895-x5fdk" event={"ID":"e5e3957c-e27b-4bb1-9ab0-cd7ecd612144","Type":"ContainerStarted","Data":"61fa1d47ba27aa82e70c68ec15187d29a3c9ee446d17936cc1abc3c7d9a053eb"}
Jul 03 10:15:00.865527 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.865409    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717895-x5fdk" event={"ID":"e5e3957c-e27b-4bb1-9ab0-cd7ecd612144","Type":"ContainerStarted","Data":"ef72911581c4e146c1b911bd32024c44f3418a756070a753c1859352fa58e82f"}
Jul 03 10:15:00.891493 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:00.891450    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29717895-x5fdk" podStartSLOduration=0.891434929 podStartE2EDuration="891.434929ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:15:00.889978086 +0000 UTC m=+1902.755266744" watchObservedRunningTime="2026-07-03 10:15:00.891434929 +0000 UTC m=+1902.756723583"
Jul 03 10:15:01.871735 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:01.871636    2571 generic.go:356] "Generic (PLEG): container finished" podID="e5e3957c-e27b-4bb1-9ab0-cd7ecd612144" containerID="61fa1d47ba27aa82e70c68ec15187d29a3c9ee446d17936cc1abc3c7d9a053eb" exitCode=0
Jul 03 10:15:01.871735 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:01.871701    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717895-x5fdk" event={"ID":"e5e3957c-e27b-4bb1-9ab0-cd7ecd612144","Type":"ContainerDied","Data":"61fa1d47ba27aa82e70c68ec15187d29a3c9ee446d17936cc1abc3c7d9a053eb"}
Jul 03 10:15:03.007062 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:03.007040    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717895-x5fdk"
Jul 03 10:15:03.070843 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:03.070800    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gw9t6\" (UniqueName: \"kubernetes.io/projected/e5e3957c-e27b-4bb1-9ab0-cd7ecd612144-kube-api-access-gw9t6\") pod \"e5e3957c-e27b-4bb1-9ab0-cd7ecd612144\" (UID: \"e5e3957c-e27b-4bb1-9ab0-cd7ecd612144\") "
Jul 03 10:15:03.072900 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:03.072876    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5e3957c-e27b-4bb1-9ab0-cd7ecd612144-kube-api-access-gw9t6" (OuterVolumeSpecName: "kube-api-access-gw9t6") pod "e5e3957c-e27b-4bb1-9ab0-cd7ecd612144" (UID: "e5e3957c-e27b-4bb1-9ab0-cd7ecd612144"). InnerVolumeSpecName "kube-api-access-gw9t6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 10:15:03.171782 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:03.171752    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gw9t6\" (UniqueName: \"kubernetes.io/projected/e5e3957c-e27b-4bb1-9ab0-cd7ecd612144-kube-api-access-gw9t6\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 10:15:03.882528 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:03.882501    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717895-x5fdk"
Jul 03 10:15:03.882699 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:03.882535    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717895-x5fdk" event={"ID":"e5e3957c-e27b-4bb1-9ab0-cd7ecd612144","Type":"ContainerDied","Data":"ef72911581c4e146c1b911bd32024c44f3418a756070a753c1859352fa58e82f"}
Jul 03 10:15:03.882699 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:03.882575    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef72911581c4e146c1b911bd32024c44f3418a756070a753c1859352fa58e82f"
Jul 03 10:15:05.219752 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:05.219716    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:05.319400 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:05.319369    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:11.597428 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:11.597394    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:11.960644 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:11.960609    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:12.246233 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:12.246156    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:14.899022 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:14.898989    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:15.022359 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:15.022325    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:15.122238 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:15.122204    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:15.261566 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:15.261484    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:15.591184 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:15.591095    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:15.686502 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:15.686465    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:19.389765 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:19.389725    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:21.803513 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:21.803476    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:15:32.116356 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:32.116320    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:32.237003 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:32.236965    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:36.977987 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:36.977956    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:37.048320 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:37.048286    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:45.562918 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:45.562887    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:15:48.719819 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:15:48.719784    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:16:04.802232 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:16:04.802195    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:16:07.195678 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:16:07.195643    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:16:07.563622 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:16:07.563538    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:16:13.276448 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:16:13.276369    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:16:17.317541 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:16:17.317504    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:16:18.356189 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:16:18.356154    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:16:54.618913 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:16:54.618875    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:16:57.482468 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:16:57.482431    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:16:57.806462 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:16:57.806387    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:17:01.749367 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:01.749335    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:07.412076 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:07.412043    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:07.490058 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:07.490027    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:07.736757 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:07.736670    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:07.946877 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:07.946845    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:09.016891 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:09.016826    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:09.144148 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:09.144115    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:09.474541 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:09.474508    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:09.922977 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:09.922942    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:10.022637 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:10.022602    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:10.123312 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:10.123282    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:11.514071 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:11.514033    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:11.903849 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:11.903806    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:17:17.536151 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:17.536113    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:17.635688 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:17.635653    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:18.146380 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:18.146337    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:24.219423 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:24.219381    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:24.326249 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:24.326210    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:24.432409 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:24.432371    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:25.144757 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:25.144717    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:17:31.804026 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:31.803990    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:34.937102 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:34.937062    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:35.284435 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:35.284348    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:43.787285 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:43.787249    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:44.392896 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:44.392854    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:44.472099 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:44.472063    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:17:56.282947 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:17:56.282914    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:02.341174 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:02.341142    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:02.440240 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:02.440207    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:02.702297 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:02.702263    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:03.045555 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:03.045464    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:03.397905 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:03.397868    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:05.742538 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:05.742507    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:05.846319 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:05.846285    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:05.963343 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:05.963310    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:06.267780 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:06.267749    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:06.485684 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:06.485652    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:06.584874 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:06.584778    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:10.245957 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:10.245907    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:10.562723 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:10.562639    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:10.866916 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:10.866814    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:11.217888 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:11.217853    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:12.815643 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:12.815608    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:18:13.886989 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:13.886957    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:14.002896 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:14.002855    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:14.154045 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:14.154012    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:14.298871 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:14.298780    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:14.740458 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:14.740412    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:14.843180 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:14.843144    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:18.329090 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:18.329025    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:18.782738 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:18.782711    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 10:18:18.789412 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:18.789388    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 10:18:22.096455 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:22.096417    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:18:27.384757 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:27.384719    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:27.479559 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:27.479524    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:35.483252 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:35.483220    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:35.583768 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:35.583738    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:42.856065 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:42.856026    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:18:47.424567 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:18:47.424530    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:19:12.956232 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:19:12.956146    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:19:17.314921 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:19:17.314892    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:19:30.100256 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:19:30.100211    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:19:32.390468 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:19:32.390435    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:19:33.376017 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:19:33.375978    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:19:33.542232 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:19:33.542193    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:19:33.908801 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:19:33.908754    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:19:34.060171 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:19:34.060135    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:19:34.160560 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:19:34.160452    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:19:34.448891 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:19:34.448735    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:19:39.060199 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:19:39.060156    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:19:45.913243 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:19:45.913209    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:20:16.101401 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:16.101367    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:16.377760 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:16.377674    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:16.630338 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:16.630250    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:16.800394 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:16.800359    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:17.218169 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:17.218132    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:17.316537 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:17.316505    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:17.413463 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:17.413425    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:17.585581 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:17.585500    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:19.412602 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:19.412562    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:20:23.803397 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:23.803364    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:23.890044 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:23.890015    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:24.114410 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:24.114326    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:24.397187 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:24.397150    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:24.708756 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:24.708669    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:25.048948 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:25.048867    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:25.204191 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:25.204134    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:28.335648 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:28.335613    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:28.423018 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:28.422982    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:28.892052 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:28.892021    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:28.992475 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:28.992438    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:29.087663 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:29.087634    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:29.193037 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:29.192997    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:32.607249 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:32.607212    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:20:32.821242 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:32.821194    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:32.933504 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:32.933464    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:33.038466 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:33.038434    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:33.271084 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:33.270965    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:33.581438 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:33.581342    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:33.788239 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:33.788206    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:36.557069 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:36.557031    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:36.643746 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:36.643704    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:36.761113 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:36.761076    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:36.988092 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:36.988060    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:37.365133 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:37.365041    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:37.469979 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:37.469947    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:41.025027 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:41.024991    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:43.603103 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:43.603069    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:20:49.090807 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:49.090771    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:49.454355 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:49.454313    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:49.561478 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:49.561444    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:49.660793 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:49.660762    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:49.836791 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:49.836712    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:54.658485 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:54.658449    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:55.005517 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:55.005418    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:20:55.541416 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:55.541376    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:55.640701 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:55.640668    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:55.718750 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:55.718716    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:55.919579 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:55.919529    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:56.116666 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:56.116630    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:56.476599 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:56.476569    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:56.577476 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:56.577444    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:56.679441 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:56.679413    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:20:56.879228 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:20:56.879140    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:04.254224 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:04.254191    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:04.396827 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:04.396783    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:04.666194 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:04.666164    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:04.819438 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:04.819408    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:05.196488 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:05.196448    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:05.468526 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:05.468444    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:21:05.659415 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:05.659376    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:05.915179 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:05.915147    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:08.570691 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:08.570642    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:08.703498 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:08.703451    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:08.796755 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:08.796720    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:09.042848 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:09.042804    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:09.404387 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:09.404356    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:09.506933 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:09.506898    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:13.031749 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:13.031714    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:13.402078 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:13.402047    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:13.719093 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:13.719010    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:13.968420 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:13.968388    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:16.506497 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:16.506461    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:21:16.821915 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:16.821809    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:16.967065 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:16.967027    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:17.124364 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:17.124274    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:17.324862 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:17.324802    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:17.854097 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:17.854061    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:17.960906 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:17.960875    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:21.295662 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:21.295628    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:27.104333 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:27.104293    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:21:31.509979 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:31.509950    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:31.611311 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:31.611278    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:33.152358 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:33.152303    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:33.213700 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:33.213669    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:35.099673 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:35.099636    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:37.998239 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:37.998207    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:21:38.613372 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:38.613334    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:21:38.713910 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:21:38.713879    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:22:08.204413 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:22:08.204365    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:22:16.550267 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:22:16.550182    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:22:20.000158 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:22:20.000124    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:22:40.106744 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:22:40.106706    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:22:55.415536 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:22:55.415498    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:23:12.714316 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:23:12.714279    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:23:18.818022 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:23:18.817995    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 10:23:18.824489 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:23:18.824465    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 10:23:28.606997 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:23:28.606966    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:23:37.931498 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:23:37.931454    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:23:40.915577 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:23:40.915475    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:23:40.981249 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:23:40.981189    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:23:42.205411 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:23:42.205371    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:23:58.022326 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:23:58.022291    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:24:09.823034 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:24:09.822994    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:24:10.725343 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:24:10.725308    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:24:14.919329 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:24:14.919293    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:24:25.259967 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:24:25.259936    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:24:28.215432 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:24:28.215393    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:24:43.130648 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:24:43.130613    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:25:06.841085 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:06.841050    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:25:11.004471 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:11.004383    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:25:24.835874 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:24.835815    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:25:33.870233 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:33.870194    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:25:33.971919 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:33.971891    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:25:43.316292 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:43.316260    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:25:49.051389 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:49.051350    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:25:49.949085 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:49.949053    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:25:53.008845 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.008798    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"]
Jul 03 10:25:53.009311 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.009246    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5e3957c-e27b-4bb1-9ab0-cd7ecd612144" containerName="cleanup"
Jul 03 10:25:53.009311 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.009259    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5e3957c-e27b-4bb1-9ab0-cd7ecd612144" containerName="cleanup"
Jul 03 10:25:53.009393 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.009336    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5e3957c-e27b-4bb1-9ab0-cd7ecd612144" containerName="cleanup"
Jul 03 10:25:53.012460 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.012436    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.015068 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.015040    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1fc47c\"/\"openshift-service-ca.crt\""
Jul 03 10:25:53.015918 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.015896    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1fc47c\"/\"rate-test-model-1fc47c-kserve-self-signed-certs\""
Jul 03 10:25:53.016044 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.015950    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1fc47c\"/\"default-dockercfg-46qkc\""
Jul 03 10:25:53.016044 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.015954    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1fc47c\"/\"kube-root-ca.crt\""
Jul 03 10:25:53.024803 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.024778    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"]
Jul 03 10:25:53.038199 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.038172    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-home\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.038337 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.038283    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p296g\" (UniqueName: \"kubernetes.io/projected/ea067670-0af5-41d1-9fd3-e9590aba1d4e-kube-api-access-p296g\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.038337 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.038316    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-model-cache\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.038425 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.038365    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-tmp-dir\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.038425 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.038391    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea067670-0af5-41d1-9fd3-e9590aba1d4e-tls-certs\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.038535 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.038455    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-dshm\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.058512 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.058485    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:25:53.139685 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.139659    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-tmp-dir\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.139877 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.139689    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea067670-0af5-41d1-9fd3-e9590aba1d4e-tls-certs\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.139877 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.139710    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-dshm\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.139877 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.139739    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-home\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.139877 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.139827    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p296g\" (UniqueName: \"kubernetes.io/projected/ea067670-0af5-41d1-9fd3-e9590aba1d4e-kube-api-access-p296g\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.140112 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.139885    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-model-cache\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.140172 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.140129    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-tmp-dir\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.140379 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.140356    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-home\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.140540 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.140467    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-model-cache\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.142847 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.142810    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea067670-0af5-41d1-9fd3-e9590aba1d4e-tls-certs\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.143053 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.143035    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-dshm\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.149239 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.149211    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p296g\" (UniqueName: \"kubernetes.io/projected/ea067670-0af5-41d1-9fd3-e9590aba1d4e-kube-api-access-p296g\") pod \"rate-test-model-1fc47c-kserve-56786b68b7-d98xk\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") " pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.323682 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.323601    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:25:53.463936 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:53.463905    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"]
Jul 03 10:25:53.466239 ip-10-0-138-129 kubenswrapper[2571]: W0703 10:25:53.466209    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea067670_0af5_41d1_9fd3_e9590aba1d4e.slice/crio-1ed8fedec47d2a3197f57a0bc1317db7747ed61f7e359cf3dc773bffea807a2c WatchSource:0}: Error finding container 1ed8fedec47d2a3197f57a0bc1317db7747ed61f7e359cf3dc773bffea807a2c: Status 404 returned error can't find the container with id 1ed8fedec47d2a3197f57a0bc1317db7747ed61f7e359cf3dc773bffea807a2c
Jul 03 10:25:54.251773 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:54.251736    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk" event={"ID":"ea067670-0af5-41d1-9fd3-e9590aba1d4e","Type":"ContainerStarted","Data":"d31863f5ad83516752d2999cea1d14d26e12db63157622ca2fef884782ff9759"}
Jul 03 10:25:54.251773 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:54.251777    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk" event={"ID":"ea067670-0af5-41d1-9fd3-e9590aba1d4e","Type":"ContainerStarted","Data":"1ed8fedec47d2a3197f57a0bc1317db7747ed61f7e359cf3dc773bffea807a2c"}
Jul 03 10:25:54.311263 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:25:54.311202    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk" podStartSLOduration=2.311183972 podStartE2EDuration="2.311183972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:25:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:25:54.310118635 +0000 UTC m=+2556.175407280" watchObservedRunningTime="2026-07-03 10:25:54.311183972 +0000 UTC m=+2556.176472624"
Jul 03 10:26:00.504893 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:26:00.504851    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:26:03.324288 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:26:03.324256    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:26:03.324709 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:26:03.324341    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:26:03.336570 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:26:03.336533    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:26:04.314610 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:26:04.314579    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:26:15.259350 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:26:15.259315    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:26:19.213011 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:26:19.212977    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:26:32.477611 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:26:32.477575    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:26:32.578449 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:26:32.578418    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:26:39.366327 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:26:39.366291    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:26:39.469556 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:26:39.469516    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:26:55.432509 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:26:55.432469    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:27:09.515783 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:09.515741    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:27:16.083305 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:16.083265    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"]
Jul 03 10:27:16.123296 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:16.123261    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:27:17.432312 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:17.432276    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:27:18.069948 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:18.069913    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:27:18.462020 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:18.461990    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:27:18.590215 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:18.590182    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:27:18.706501 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:18.706456    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:27:18.907085 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:18.907050    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:27:22.404500 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:22.404460    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:27:23.533879 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:23.533847    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:27:31.103765 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.103710    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk" podUID="ea067670-0af5-41d1-9fd3-e9590aba1d4e" containerName="main" containerID="cri-o://d31863f5ad83516752d2999cea1d14d26e12db63157622ca2fef884782ff9759" gracePeriod=45
Jul 03 10:27:31.356514 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.356442    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:27:31.434241 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.434207    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p296g\" (UniqueName: \"kubernetes.io/projected/ea067670-0af5-41d1-9fd3-e9590aba1d4e-kube-api-access-p296g\") pod \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") "
Jul 03 10:27:31.434420 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.434294    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea067670-0af5-41d1-9fd3-e9590aba1d4e-tls-certs\") pod \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") "
Jul 03 10:27:31.434420 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.434323    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-model-cache\") pod \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") "
Jul 03 10:27:31.434420 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.434361    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-dshm\") pod \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") "
Jul 03 10:27:31.434420 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.434388    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-home\") pod \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") "
Jul 03 10:27:31.434420 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.434416    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-tmp-dir\") pod \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\" (UID: \"ea067670-0af5-41d1-9fd3-e9590aba1d4e\") "
Jul 03 10:27:31.434685 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.434585    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-model-cache" (OuterVolumeSpecName: "model-cache") pod "ea067670-0af5-41d1-9fd3-e9590aba1d4e" (UID: "ea067670-0af5-41d1-9fd3-e9590aba1d4e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 10:27:31.434742 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.434692    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-home" (OuterVolumeSpecName: "home") pod "ea067670-0af5-41d1-9fd3-e9590aba1d4e" (UID: "ea067670-0af5-41d1-9fd3-e9590aba1d4e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 10:27:31.434791 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.434758    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ea067670-0af5-41d1-9fd3-e9590aba1d4e" (UID: "ea067670-0af5-41d1-9fd3-e9590aba1d4e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 10:27:31.434791 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.434780    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-model-cache\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 10:27:31.434896 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.434797    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-home\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 10:27:31.436593 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.436565    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea067670-0af5-41d1-9fd3-e9590aba1d4e-kube-api-access-p296g" (OuterVolumeSpecName: "kube-api-access-p296g") pod "ea067670-0af5-41d1-9fd3-e9590aba1d4e" (UID: "ea067670-0af5-41d1-9fd3-e9590aba1d4e"). InnerVolumeSpecName "kube-api-access-p296g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 10:27:31.436593 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.436578    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-dshm" (OuterVolumeSpecName: "dshm") pod "ea067670-0af5-41d1-9fd3-e9590aba1d4e" (UID: "ea067670-0af5-41d1-9fd3-e9590aba1d4e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 10:27:31.436593 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.436570    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea067670-0af5-41d1-9fd3-e9590aba1d4e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ea067670-0af5-41d1-9fd3-e9590aba1d4e" (UID: "ea067670-0af5-41d1-9fd3-e9590aba1d4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 10:27:31.535279 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.535240    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p296g\" (UniqueName: \"kubernetes.io/projected/ea067670-0af5-41d1-9fd3-e9590aba1d4e-kube-api-access-p296g\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 10:27:31.535279 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.535272    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea067670-0af5-41d1-9fd3-e9590aba1d4e-tls-certs\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 10:27:31.535279 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.535281    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-dshm\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 10:27:31.535279 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.535293    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea067670-0af5-41d1-9fd3-e9590aba1d4e-tmp-dir\") on node \"ip-10-0-138-129.ec2.internal\" DevicePath \"\""
Jul 03 10:27:31.760823 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.760788    2571 generic.go:356] "Generic (PLEG): container finished" podID="ea067670-0af5-41d1-9fd3-e9590aba1d4e" containerID="d31863f5ad83516752d2999cea1d14d26e12db63157622ca2fef884782ff9759" exitCode=0
Jul 03 10:27:31.761037 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.760863    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"
Jul 03 10:27:31.761037 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.760879    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk" event={"ID":"ea067670-0af5-41d1-9fd3-e9590aba1d4e","Type":"ContainerDied","Data":"d31863f5ad83516752d2999cea1d14d26e12db63157622ca2fef884782ff9759"}
Jul 03 10:27:31.761037 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.760928    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk" event={"ID":"ea067670-0af5-41d1-9fd3-e9590aba1d4e","Type":"ContainerDied","Data":"1ed8fedec47d2a3197f57a0bc1317db7747ed61f7e359cf3dc773bffea807a2c"}
Jul 03 10:27:31.761037 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.760949    2571 scope.go:117] "RemoveContainer" containerID="d31863f5ad83516752d2999cea1d14d26e12db63157622ca2fef884782ff9759"
Jul 03 10:27:31.770746 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.770721    2571 scope.go:117] "RemoveContainer" containerID="d31863f5ad83516752d2999cea1d14d26e12db63157622ca2fef884782ff9759"
Jul 03 10:27:31.771040 ip-10-0-138-129 kubenswrapper[2571]: E0703 10:27:31.771024    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d31863f5ad83516752d2999cea1d14d26e12db63157622ca2fef884782ff9759\": container with ID starting with d31863f5ad83516752d2999cea1d14d26e12db63157622ca2fef884782ff9759 not found: ID does not exist" containerID="d31863f5ad83516752d2999cea1d14d26e12db63157622ca2fef884782ff9759"
Jul 03 10:27:31.771111 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.771047    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d31863f5ad83516752d2999cea1d14d26e12db63157622ca2fef884782ff9759"} err="failed to get container status \"d31863f5ad83516752d2999cea1d14d26e12db63157622ca2fef884782ff9759\": rpc error: code = NotFound desc = could not find container \"d31863f5ad83516752d2999cea1d14d26e12db63157622ca2fef884782ff9759\": container with ID starting with d31863f5ad83516752d2999cea1d14d26e12db63157622ca2fef884782ff9759 not found: ID does not exist"
Jul 03 10:27:31.783047 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.783020    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"]
Jul 03 10:27:31.786758 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:31.786734    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1fc47c/rate-test-model-1fc47c-kserve-56786b68b7-d98xk"]
Jul 03 10:27:32.765754 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:27:32.765725    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea067670-0af5-41d1-9fd3-e9590aba1d4e" path="/var/lib/kubelet/pods/ea067670-0af5-41d1-9fd3-e9590aba1d4e/volumes"
Jul 03 10:28:01.170168 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:01.170127    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:01.588774 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:01.588680    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:01.740874 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:01.740817    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:01.919139 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:01.919105    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:02.394983 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:02.394898    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:02.491632 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:02.491598    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:02.591894 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:02.591825    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:02.746639 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:02.746609    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:05.616791 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:05.613912    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lnjn"]
Jul 03 10:28:09.054048 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:09.054013    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:09.153626 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:09.153584    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:09.332251 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:09.332164    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:09.745038 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:09.745010    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:10.100958 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:10.100875    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6cc8dc5447-m97dg_ec6aae00-836b-4613-a5a5-a0a5b91c4edf/authorino/0.log"
Jul 03 10:28:14.783420 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:14.783389    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-llfzz_482fd11c-ac13-482a-befc-ab34e0d98a21/manager/0.log"
Jul 03 10:28:14.901952 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:14.901921    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9xrx7_0cee01c2-e4cf-4c25-8337-293f81074e2b/manager/0.log"
Jul 03 10:28:15.170112 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:15.170083    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717880-j5nbw_7c319aa1-3f7f-40a5-a4a3-f938749d4103/cleanup/0.log"
Jul 03 10:28:15.276974 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:15.276945    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717895-x5fdk_e5e3957c-e27b-4bb1-9ab0-cd7ecd612144/cleanup/0.log"
Jul 03 10:28:15.545146 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:15.545069    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mtgkq_b1ae1da7-3d46-482e-9554-2cf34350243e/server/0.log"
Jul 03 10:28:15.662638 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:15.662608    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-p5rm4_ef02fb74-8757-4753-a979-c926bcfb7b9f/manager/0.log"
Jul 03 10:28:15.775104 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:15.775076    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6_a7cd9fda-9cec-4479-a0ac-779a5e0a38e0/copy-manifests/0.log"
Jul 03 10:28:15.782730 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:15.782702    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6_a7cd9fda-9cec-4479-a0ac-779a5e0a38e0/manager/0.log"
Jul 03 10:28:17.607879 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:17.607366    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6cc8dc5447-m97dg_ec6aae00-836b-4613-a5a5-a0a5b91c4edf/authorino/0.log"
Jul 03 10:28:17.729822 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:17.729787    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6wng7_b1d9c001-990a-4aae-abd0-807b117ddd46/manager/0.log"
Jul 03 10:28:17.844397 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:17.844366    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7qhwb_a200bf34-d3ed-409b-8104-4ca1384c07a0/manager/0.log"
Jul 03 10:28:17.954499 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:17.954466    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-x8ktz_3b350445-4582-4d07-a021-256d60bb143e/kuadrant-console-plugin/0.log"
Jul 03 10:28:18.080665 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:18.080630    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6xtc7_42019489-0140-4d95-b62d-7ee8a0e1abc2/registry-server/0.log"
Jul 03 10:28:18.214730 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:18.214646    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wth9j_e4b86c35-c428-4375-901f-2c470e7621a8/manager/0.log"
Jul 03 10:28:18.353937 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:18.353910    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2lnjn_84b5540a-d878-43fd-9154-3d431171bf7c/limitador/0.log"
Jul 03 10:28:18.495996 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:18.495917    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-x4qtq_7848c387-6f31-4a59-b5cd-be0a154dbf04/manager/0.log"
Jul 03 10:28:18.852655 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:18.852577    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 10:28:18.858882 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:18.858862    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hmsd_d915f962-809c-4394-aa7b-72b49fdde9b2/ovn-acl-logging/0.log"
Jul 03 10:28:19.124476 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:19.124381    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f5df5b9bf-hknrl_0ed2ca98-af98-40cc-a837-12963a215d81/kube-auth-proxy/0.log"
Jul 03 10:28:19.769148 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:19.769122    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c5cd9c95-m6fvt_865a12bb-5687-4bc6-8f85-8a5df5db37fa/router/0.log"
Jul 03 10:28:20.241923 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:20.241890    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-n5rq2_8a8b170b-8cdb-44a2-94dc-833e5450faf7/main/0.log"
Jul 03 10:28:24.963479 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:24.963441    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sldrw/must-gather-v242c"]
Jul 03 10:28:24.963901 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:24.963877    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea067670-0af5-41d1-9fd3-e9590aba1d4e" containerName="main"
Jul 03 10:28:24.963901 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:24.963889    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea067670-0af5-41d1-9fd3-e9590aba1d4e" containerName="main"
Jul 03 10:28:24.964001 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:24.963987    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea067670-0af5-41d1-9fd3-e9590aba1d4e" containerName="main"
Jul 03 10:28:24.968597 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:24.968574    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sldrw/must-gather-v242c"
Jul 03 10:28:24.971146 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:24.971088    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:24.971329 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:24.971190    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sldrw\"/\"openshift-service-ca.crt\""
Jul 03 10:28:24.971450 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:24.971431    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sldrw\"/\"default-dockercfg-2snr6\""
Jul 03 10:28:24.971512 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:24.971463    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sldrw\"/\"kube-root-ca.crt\""
Jul 03 10:28:24.977230 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:24.977203    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sldrw/must-gather-v242c"]
Jul 03 10:28:25.022342 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:25.022299    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/95022a62-640c-406b-8aae-e6295a64159e-must-gather-output\") pod \"must-gather-v242c\" (UID: \"95022a62-640c-406b-8aae-e6295a64159e\") " pod="openshift-must-gather-sldrw/must-gather-v242c"
Jul 03 10:28:25.022522 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:25.022345    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p95sj\" (UniqueName: \"kubernetes.io/projected/95022a62-640c-406b-8aae-e6295a64159e-kube-api-access-p95sj\") pod \"must-gather-v242c\" (UID: \"95022a62-640c-406b-8aae-e6295a64159e\") " pod="openshift-must-gather-sldrw/must-gather-v242c"
Jul 03 10:28:25.073729 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:25.073679    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024
Jul 03 10:28:25.123646 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:25.123610    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/95022a62-640c-406b-8aae-e6295a64159e-must-gather-output\") pod \"must-gather-v242c\" (UID: \"95022a62-640c-406b-8aae-e6295a64159e\") " pod="openshift-must-gather-sldrw/must-gather-v242c"
Jul 03 10:28:25.123646 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:25.123643    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p95sj\" (UniqueName: \"kubernetes.io/projected/95022a62-640c-406b-8aae-e6295a64159e-kube-api-access-p95sj\") pod \"must-gather-v242c\" (UID: \"95022a62-640c-406b-8aae-e6295a64159e\") " pod="openshift-must-gather-sldrw/must-gather-v242c"
Jul 03 10:28:25.123973 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:25.123951    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/95022a62-640c-406b-8aae-e6295a64159e-must-gather-output\") pod \"must-gather-v242c\" (UID: \"95022a62-640c-406b-8aae-e6295a64159e\") " pod="openshift-must-gather-sldrw/must-gather-v242c"
Jul 03 10:28:25.136612 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:25.136582    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p95sj\" (UniqueName: \"kubernetes.io/projected/95022a62-640c-406b-8aae-e6295a64159e-kube-api-access-p95sj\") pod \"must-gather-v242c\" (UID: \"95022a62-640c-406b-8aae-e6295a64159e\") " pod="openshift-must-gather-sldrw/must-gather-v242c"
Jul 03 10:28:25.279738 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:25.279648    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sldrw/must-gather-v242c"
Jul 03 10:28:25.447668 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:25.447645    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sldrw/must-gather-v242c"]
Jul 03 10:28:25.449782 ip-10-0-138-129 kubenswrapper[2571]: W0703 10:28:25.449748    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95022a62_640c_406b_8aae_e6295a64159e.slice/crio-58c161a6fbb5e6768a54f81dbbc77fdb93f381a7791b3ff32bb9dd19f534863b WatchSource:0}: Error finding container 58c161a6fbb5e6768a54f81dbbc77fdb93f381a7791b3ff32bb9dd19f534863b: Status 404 returned error can't find the container with id 58c161a6fbb5e6768a54f81dbbc77fdb93f381a7791b3ff32bb9dd19f534863b
Jul 03 10:28:25.451733 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:25.451705    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 10:28:26.038092 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:26.038058    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sldrw/must-gather-v242c" event={"ID":"95022a62-640c-406b-8aae-e6295a64159e","Type":"ContainerStarted","Data":"58c161a6fbb5e6768a54f81dbbc77fdb93f381a7791b3ff32bb9dd19f534863b"}
Jul 03 10:28:27.047233 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:27.047160    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sldrw/must-gather-v242c" event={"ID":"95022a62-640c-406b-8aae-e6295a64159e","Type":"ContainerStarted","Data":"8f7b01c0e332cb489d65b9ec16bfdf62d44cb942ecf802ed4bed4969883adf87"}
Jul 03 10:28:27.047233 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:27.047214    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sldrw/must-gather-v242c" event={"ID":"95022a62-640c-406b-8aae-e6295a64159e","Type":"ContainerStarted","Data":"67ef58f13f2c7b5241db39d56b33eefee852ae71e39c3d1baffe655b344b112e"}
Jul 03 10:28:27.064688 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:27.064613    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sldrw/must-gather-v242c" podStartSLOduration=1.881410089 podStartE2EDuration="3.064589155s" totalImagesPullingTime="1.183179066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:28:27.063076554 +0000 UTC m=+2708.928365220" watchObservedRunningTime="2026-07-03 10:28:27.064589155 +0000 UTC m=+2708.929877810"
Jul 03 10:28:28.395114 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:28.395083    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n4gfv_d642348a-9842-43db-bcdf-5d84ef42e828/global-pull-secret-syncer/0.log"
Jul 03 10:28:28.499130 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:28.499089    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log"
Jul 03 10:28:28.637134 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:28.637101    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-129.ec2.internal_b662888df2876571019936bda200019b/haproxy/0.log"
Jul 03 10:28:33.857823 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:33.857414    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6cc8dc5447-m97dg_ec6aae00-836b-4613-a5a5-a0a5b91c4edf/authorino/0.log"
Jul 03 10:28:33.914856 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:33.914501    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6wng7_b1d9c001-990a-4aae-abd0-807b117ddd46/manager/0.log"
Jul 03 10:28:33.981807 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:33.981770    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7qhwb_a200bf34-d3ed-409b-8104-4ca1384c07a0/manager/0.log"
Jul 03 10:28:34.062184 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:34.062067    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-x8ktz_3b350445-4582-4d07-a021-256d60bb143e/kuadrant-console-plugin/0.log"
Jul 03 10:28:34.158914 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:34.158880    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6xtc7_42019489-0140-4d95-b62d-7ee8a0e1abc2/registry-server/0.log"
Jul 03 10:28:34.424903 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:34.424733    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wth9j_e4b86c35-c428-4375-901f-2c470e7621a8/manager/0.log"
Jul 03 10:28:34.481280 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:34.481250    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2lnjn_84b5540a-d878-43fd-9154-3d431171bf7c/limitador/0.log"
Jul 03 10:28:34.560605 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:34.560553    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-x4qtq_7848c387-6f31-4a59-b5cd-be0a154dbf04/manager/0.log"
Jul 03 10:28:36.425613 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:36.425577    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7966bfc5ff-8gnwj_93117215-25d2-4b44-8977-47a0363088ab/metrics-server/0.log"
Jul 03 10:28:36.455198 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:36.455164    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-77pdg_8b6a5cdc-691b-4621-bf36-133f421c85cc/monitoring-plugin/0.log"
Jul 03 10:28:36.570303 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:36.570275    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dgqxn_91401011-fe57-4baf-80d2-2d4e813fb10e/node-exporter/0.log"
Jul 03 10:28:36.597153 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:36.597119    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dgqxn_91401011-fe57-4baf-80d2-2d4e813fb10e/kube-rbac-proxy/0.log"
Jul 03 10:28:36.620935 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:36.620892    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dgqxn_91401011-fe57-4baf-80d2-2d4e813fb10e/init-textfile/0.log"
Jul 03 10:28:36.751604 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:36.751528    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-lc59n_d5f267cc-926a-4c7d-bcf6-f1aad226c0d4/kube-rbac-proxy-main/0.log"
Jul 03 10:28:36.777234 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:36.777188    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-lc59n_d5f267cc-926a-4c7d-bcf6-f1aad226c0d4/kube-rbac-proxy-self/0.log"
Jul 03 10:28:36.805590 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:36.805558    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-lc59n_d5f267cc-926a-4c7d-bcf6-f1aad226c0d4/openshift-state-metrics/0.log"
Jul 03 10:28:37.061817 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:37.061730    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-zvhxp_34415362-51c1-41fd-a525-7520f25f65e8/prometheus-operator/0.log"
Jul 03 10:28:37.085767 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:37.085735    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-zvhxp_34415362-51c1-41fd-a525-7520f25f65e8/kube-rbac-proxy/0.log"
Jul 03 10:28:37.122729 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:37.122698    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-q66kf_c2584a2c-1655-4547-91a1-ef907627535a/prometheus-operator-admission-webhook/0.log"
Jul 03 10:28:37.317911 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:37.317816    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-596fdd74fc-lgvlc_47d7cde5-4688-4610-abd4-de8ff576fd75/thanos-query/0.log"
Jul 03 10:28:37.352288 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:37.352262    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-596fdd74fc-lgvlc_47d7cde5-4688-4610-abd4-de8ff576fd75/kube-rbac-proxy-web/0.log"
Jul 03 10:28:37.377026 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:37.376962    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-596fdd74fc-lgvlc_47d7cde5-4688-4610-abd4-de8ff576fd75/kube-rbac-proxy/0.log"
Jul 03 10:28:37.407025 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:37.406997    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-596fdd74fc-lgvlc_47d7cde5-4688-4610-abd4-de8ff576fd75/prom-label-proxy/0.log"
Jul 03 10:28:37.437562 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:37.437521    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-596fdd74fc-lgvlc_47d7cde5-4688-4610-abd4-de8ff576fd75/kube-rbac-proxy-rules/0.log"
Jul 03 10:28:37.461528 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:37.461496    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-596fdd74fc-lgvlc_47d7cde5-4688-4610-abd4-de8ff576fd75/kube-rbac-proxy-metrics/0.log"
Jul 03 10:28:38.258195 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.258150    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"]
Jul 03 10:28:38.266827 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.266801    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.273742 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.273702    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"]
Jul 03 10:28:38.376018 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.375966    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a845a66a-105f-4e9c-812b-ba363c9b1ff6-sys\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.376195 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.376034    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a845a66a-105f-4e9c-812b-ba363c9b1ff6-proc\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.376195 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.376060    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a845a66a-105f-4e9c-812b-ba363c9b1ff6-podres\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.376330 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.376239    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a845a66a-105f-4e9c-812b-ba363c9b1ff6-lib-modules\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.376330 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.376280    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6s7b\" (UniqueName: \"kubernetes.io/projected/a845a66a-105f-4e9c-812b-ba363c9b1ff6-kube-api-access-t6s7b\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.476857 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.476799    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a845a66a-105f-4e9c-812b-ba363c9b1ff6-sys\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.477351 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.476884    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a845a66a-105f-4e9c-812b-ba363c9b1ff6-proc\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.477351 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.476914    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a845a66a-105f-4e9c-812b-ba363c9b1ff6-podres\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.477351 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.476931    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a845a66a-105f-4e9c-812b-ba363c9b1ff6-sys\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.477351 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.477027    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a845a66a-105f-4e9c-812b-ba363c9b1ff6-lib-modules\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.477351 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.477057    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6s7b\" (UniqueName: \"kubernetes.io/projected/a845a66a-105f-4e9c-812b-ba363c9b1ff6-kube-api-access-t6s7b\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.477351 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.477076    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a845a66a-105f-4e9c-812b-ba363c9b1ff6-podres\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.477351 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.477228    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a845a66a-105f-4e9c-812b-ba363c9b1ff6-proc\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.477351 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.477329    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a845a66a-105f-4e9c-812b-ba363c9b1ff6-lib-modules\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.494334 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.494303    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6s7b\" (UniqueName: \"kubernetes.io/projected/a845a66a-105f-4e9c-812b-ba363c9b1ff6-kube-api-access-t6s7b\") pod \"perf-node-gather-daemonset-ggj5z\" (UID: \"a845a66a-105f-4e9c-812b-ba363c9b1ff6\") " pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.585696 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.585594    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:38.784793 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:38.784753    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"]
Jul 03 10:28:39.173510 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:39.173462    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z" event={"ID":"a845a66a-105f-4e9c-812b-ba363c9b1ff6","Type":"ContainerStarted","Data":"08f6db50fcdd5aebd6701d81fd3fdacd193ed02aaa0397d64a64f4c5162838d9"}
Jul 03 10:28:39.173712 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:39.173583    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z" event={"ID":"a845a66a-105f-4e9c-812b-ba363c9b1ff6","Type":"ContainerStarted","Data":"adab83f3d2f34ebe774b3971de89a50942e481bda9c1842e5aed05ce0038c3bb"}
Jul 03 10:28:39.173712 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:39.173624    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:39.191768 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:39.191707    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z" podStartSLOduration=1.1916866879999999 podStartE2EDuration="1.191686688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:28:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:28:39.19081186 +0000 UTC m=+2721.056100519" watchObservedRunningTime="2026-07-03 10:28:39.191686688 +0000 UTC m=+2721.056975344"
Jul 03 10:28:39.918823 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:39.918793    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-784df47f47-jmxnc_488ce27b-8d7a-495f-84be-e568e33eb263/console/0.log"
Jul 03 10:28:40.518426 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:40.518400    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-shl4s_901fcf7f-0f02-41d9-a471-55471d7ecbe7/volume-data-source-validator/0.log"
Jul 03 10:28:41.716142 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:41.716114    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wbkqf_889943a6-a12d-4476-ba1e-d0fcc7de0f42/dns/0.log"
Jul 03 10:28:41.738296 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:41.738262    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wbkqf_889943a6-a12d-4476-ba1e-d0fcc7de0f42/kube-rbac-proxy/0.log"
Jul 03 10:28:41.807277 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:41.807248    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gmjss_20bc336f-ff0c-4072-8275-46857fb9f016/dns-node-resolver/0.log"
Jul 03 10:28:42.421492 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:42.421464    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fng7l_dd3cac63-b412-453b-90bc-5b7897a9920f/node-ca/0.log"
Jul 03 10:28:43.661989 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:43.661962    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f5df5b9bf-hknrl_0ed2ca98-af98-40cc-a837-12963a215d81/kube-auth-proxy/0.log"
Jul 03 10:28:43.866751 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:43.866721    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c5cd9c95-m6fvt_865a12bb-5687-4bc6-8f85-8a5df5db37fa/router/0.log"
Jul 03 10:28:44.650556 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:44.650527    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kqx5h_5fc5c4fd-aaba-4149-86aa-ded1a82da601/serve-healthcheck-canary/0.log"
Jul 03 10:28:45.195717 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:45.195683    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sldrw/perf-node-gather-daemonset-ggj5z"
Jul 03 10:28:45.326183 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:45.326157    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gz8ww_7f1108aa-d8c6-4608-aa94-a85740bfdc1e/kube-rbac-proxy/0.log"
Jul 03 10:28:45.349182 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:45.349146    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gz8ww_7f1108aa-d8c6-4608-aa94-a85740bfdc1e/exporter/0.log"
Jul 03 10:28:45.370504 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:45.370479    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gz8ww_7f1108aa-d8c6-4608-aa94-a85740bfdc1e/extractor/0.log"
Jul 03 10:28:47.734368 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:47.734335    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-llfzz_482fd11c-ac13-482a-befc-ab34e0d98a21/manager/0.log"
Jul 03 10:28:47.914955 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:47.914915    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9xrx7_0cee01c2-e4cf-4c25-8337-293f81074e2b/manager/0.log"
Jul 03 10:28:47.972476 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:47.972443    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717880-j5nbw_7c319aa1-3f7f-40a5-a4a3-f938749d4103/cleanup/0.log"
Jul 03 10:28:48.005271 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:48.005241    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717895-x5fdk_e5e3957c-e27b-4bb1-9ab0-cd7ecd612144/cleanup/0.log"
Jul 03 10:28:48.129023 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:48.128993    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mtgkq_b1ae1da7-3d46-482e-9554-2cf34350243e/server/0.log"
Jul 03 10:28:48.219526 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:48.219451    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-p5rm4_ef02fb74-8757-4753-a979-c926bcfb7b9f/manager/0.log"
Jul 03 10:28:48.258635 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:48.258604    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6_a7cd9fda-9cec-4479-a0ac-779a5e0a38e0/manager/0.log"
Jul 03 10:28:48.281514 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:48.281483    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f66d7cdc8-bxgq6_a7cd9fda-9cec-4479-a0ac-779a5e0a38e0/copy-manifests/0.log"
Jul 03 10:28:49.324449 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:49.324418    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f5df5b9bf-hknrl_0ed2ca98-af98-40cc-a837-12963a215d81/kube-auth-proxy/0.log"
Jul 03 10:28:49.475418 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:49.475388    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c5cd9c95-m6fvt_865a12bb-5687-4bc6-8f85-8a5df5db37fa/router/0.log"
Jul 03 10:28:50.792022 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:50.791986    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f5df5b9bf-hknrl_0ed2ca98-af98-40cc-a837-12963a215d81/kube-auth-proxy/0.log"
Jul 03 10:28:50.932070 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:50.932038    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c5cd9c95-m6fvt_865a12bb-5687-4bc6-8f85-8a5df5db37fa/router/0.log"
Jul 03 10:28:55.082614 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:55.082573    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wpk2r_7bcdf3e0-7d30-4010-9219-8a8901e43f25/kube-storage-version-migrator-operator/1.log"
Jul 03 10:28:55.083979 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:55.083956    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wpk2r_7bcdf3e0-7d30-4010-9219-8a8901e43f25/kube-storage-version-migrator-operator/0.log"
Jul 03 10:28:55.721949 ip-10-0-138-129 kubenswrapper[2571]: I0703 10:28:55.721909    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g8hbm_24b56ac3-1415-4f69-aedc-44c6c2efd7dc/konnectivity-agent/0.log" limit=1024